2023-10-06T13:33:36.903 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T13:33:36.903 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609 --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} --name yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi --owner scheduled_yuriw@teuthology --verbose -- /home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609/orig.config.yaml 2023-10-06T13:33:36.943 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T13:33:37.022 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609 branch: wip-yuri7-testing-2023-10-04-1350-reef description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} email: yweinste@redhat.com first_in_suite: false job_id: '7412609' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri7-testing-2023-10-04-1350-reef ceph: conf: client: debug monc: 20 debug ms: 1 debug objecter: 20 debug rados: 20 global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 5000 ms type: async osd heartbeat grace: 80 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osd crush smoke test: false mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 mon warn on pool no app: false paxos service trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 1 debug bluestore: 1 debug monc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd class default list: '*' osd class load list: '*' osd client watch timeout: 120 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd fast shutdown: false osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - reached quota - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(SLOW_OPS\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(OBJECT_MISPLACED\) - slow request - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(SLOW_OPS\) - slow request sha1: 4af54c4a1461e3792e9783e337bb62ce429214f3 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: debuginfo: true flavor: default sha1: 4af54c4a1461e3792e9783e337bb62ce429214f3 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri7-testing-2023-10-04-1350-reef sha1: 4af54c4a1461e3792e9783e337bb62ce429214f3 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 2590 sha1: 4af54c4a1461e3792e9783e337bb62ce429214f3 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri7-testing-2023-10-04-1350-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 4af54c4a1461e3792e9783e337bb62ce429214f3 targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKFVHDP42vImDPuf04uS5rW92D9ajR1cl+iGPKEkXHSvOkbk7aW87koA584qj1YGCwIYmkWLzgrQm43lX51y5DE= smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPAnIPUznwUocjlTpOrgy72Ol5ISFYjF2B2TswcYhFAoqtpeWOZpkDxB8ufWw295eDlKNnYynEE4qzouoO9i7dk= tasks: - install: null - ceph: null - workunit: clients: client.0: - rados/test.sh env: ALLOW_TIMEOUTS: '1' timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:58:21 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T13:33:37.023 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa; will attempt to use it 2023-10-06T13:33:37.023 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks 2023-10-06T13:33:37.023 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T13:33:37.025 INFO:teuthology.task.internal:Checking packages... 2023-10-06T13:33:37.047 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '4af54c4a1461e3792e9783e337bb62ce429214f3' 2023-10-06T13:33:37.047 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T13:33:37.047 INFO:teuthology.packaging:ref: None 2023-10-06T13:33:37.047 INFO:teuthology.packaging:tag: None 2023-10-06T13:33:37.047 INFO:teuthology.packaging:branch: wip-yuri7-testing-2023-10-04-1350-reef 2023-10-06T13:33:37.047 INFO:teuthology.packaging:sha1: 4af54c4a1461e3792e9783e337bb62ce429214f3 2023-10-06T13:33:37.047 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri7-testing-2023-10-04-1350-reef 2023-10-06T13:33:37.191 INFO:teuthology.task.internal:Found packages for ceph version 18.2.0-596.g4af54c4a 2023-10-06T13:33:37.191 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T13:33:37.200 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T13:33:37.200 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T13:33:37.245 INFO:teuthology.task.internal:Saving configuration 2023-10-06T13:33:37.264 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T13:33:37.275 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T13:33:37.300 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 13:28:40.481228', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKFVHDP42vImDPuf04uS5rW92D9ajR1cl+iGPKEkXHSvOkbk7aW87koA584qj1YGCwIYmkWLzgrQm43lX51y5DE='} 2023-10-06T13:33:37.320 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 13:28:40.479983', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPAnIPUznwUocjlTpOrgy72Ol5ISFYjF2B2TswcYhFAoqtpeWOZpkDxB8ufWw295eDlKNnYynEE4qzouoO9i7dk='} 2023-10-06T13:33:37.320 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T13:33:37.329 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-10-06T13:33:37.329 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-10-06T13:33:37.329 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T13:33:37.421 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff11d92eee0>, signals=[15]) 2023-10-06T13:33:37.422 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T13:33:37.428 INFO:teuthology.task.internal:Opening connections... 2023-10-06T13:33:37.428 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2023-10-06T13:33:37.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T13:33:37.527 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2023-10-06T13:33:37.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T13:33:37.609 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T13:33:37.620 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2023-10-06T13:33:37.638 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2023-10-06T13:33:37.638 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2023-10-06T13:33:37.694 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2023-10-06T13:33:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2023-10-06T13:33:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2023-10-06T13:33:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2023-10-06T13:33:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2023-10-06T13:33:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2023-10-06T13:33:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-06T13:33:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2023-10-06T13:33:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2023-10-06T13:33:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-06T13:33:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2023-10-06T13:33:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T13:33:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-06T13:33:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-06T13:33:37.696 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2023-10-06T13:33:37.722 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2023-10-06T13:33:37.740 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2023-10-06T13:33:37.740 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2023-10-06T13:33:37.797 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2023-10-06T13:33:37.797 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2023-10-06T13:33:37.797 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2023-10-06T13:33:37.797 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2023-10-06T13:33:37.797 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2023-10-06T13:33:37.797 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2023-10-06T13:33:37.797 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-06T13:33:37.797 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2023-10-06T13:33:37.797 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2023-10-06T13:33:37.797 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-06T13:33:37.798 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2023-10-06T13:33:37.798 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T13:33:37.798 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-06T13:33:37.798 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-06T13:33:37.798 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2023-10-06T13:33:37.822 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T13:33:37.832 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T13:33:37.855 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T13:33:37.855 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2023-10-06T13:33:37.857 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2023-10-06T13:33:37.873 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T13:33:37.881 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T13:33:37.881 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2023-10-06T13:33:37.915 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2023-10-06T13:33:38.014 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T13:33:38.070 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T13:33:38.088 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-06T13:33:38.088 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-06T13:33:38.089 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T13:33:38.089 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T13:33:38.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T13:33:38.106 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2023-10-06T13:33:38.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T13:33:38.107 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2023-10-06T13:33:38.164 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-361.el9.x86_64 2023-10-06T13:33:38.164 INFO:teuthology.task.kernel:Running kernel on smithi083: 5.14.0-361.el9.x86_64 2023-10-06T13:33:38.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum install -y kernel 2023-10-06T13:33:38.166 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-361.el9.x86_64 2023-10-06T13:33:38.166 INFO:teuthology.task.kernel:Running kernel on smithi019: 5.14.0-361.el9.x86_64 2023-10-06T13:33:38.166 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2023-10-06T13:33:40.263 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-9 - AppStream 14 MB/s | 6.3 MB 00:00 2023-10-06T13:33:40.277 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS-9 - AppStream 13 MB/s | 6.3 MB 00:00 2023-10-06T13:33:41.629 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.0 MB 00:00 2023-10-06T13:33:41.635 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS-9 - BaseOS 6.3 MB/s | 2.0 MB 00:00 2023-10-06T13:33:42.429 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 7.9 MB 00:00 2023-10-06T13:33:44.009 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 7.9 MB 00:02 2023-10-06T13:33:44.338 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 21 MB/s | 18 MB 00:00 2023-10-06T13:33:45.886 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 20 MB/s | 18 MB 00:00 2023-10-06T13:33:49.239 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 5.7 MB 00:01 2023-10-06T13:33:50.831 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 5.7 MB 00:01 2023-10-06T13:33:51.124 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 14 kB 00:00 2023-10-06T13:33:52.195 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 14 kB 00:00 2023-10-06T13:33:52.510 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 15 MB/s | 19 MB 00:01 2023-10-06T13:33:54.822 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 7.6 MB/s | 19 MB 00:02 2023-10-06T13:33:57.124 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2023-10-06T13:33:59.239 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-10-06T13:33:59.260 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-10-06T13:33:59.338 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2023-10-06T13:33:59.343 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:33:59.349 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:33:59.350 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-10-06T13:33:59.350 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:33:59.350 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:33:59.350 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-373.el9 CentOS-BaseOS 5.2 M 2023-10-06T13:33:59.350 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-10-06T13:33:59.350 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core x86_64 5.14.0-373.el9 CentOS-BaseOS 20 M 2023-10-06T13:33:59.350 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules x86_64 5.14.0-373.el9 CentOS-BaseOS 37 M 2023-10-06T13:33:59.350 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core x86_64 5.14.0-373.el9 CentOS-BaseOS 32 M 2023-10-06T13:33:59.350 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:33:59.350 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:33:59.350 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:33:59.351 INFO:teuthology.orchestra.run.smithi019.stdout:Install 4 Packages 2023-10-06T13:33:59.351 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:33:59.351 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 94 M 2023-10-06T13:33:59.351 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 120 M 2023-10-06T13:33:59.351 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:33:59.709 INFO:teuthology.orchestra.run.smithi019.stdout:(1/4): kernel-5.14.0-373.el9.x86_64.rpm 15 MB/s | 5.2 MB 00:00 2023-10-06T13:34:00.184 INFO:teuthology.orchestra.run.smithi019.stdout:(2/4): kernel-core-5.14.0-373.el9.x86_64.rpm 24 MB/s | 20 MB 00:00 2023-10-06T13:34:00.726 INFO:teuthology.orchestra.run.smithi019.stdout:(3/4): kernel-modules-core-5.14.0-373.el9.x86_6 32 MB/s | 32 MB 00:01 2023-10-06T13:34:01.201 INFO:teuthology.orchestra.run.smithi019.stdout:(4/4): kernel-modules-5.14.0-373.el9.x86_64.rpm 20 MB/s | 37 MB 00:01 2023-10-06T13:34:01.263 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:34:01.263 INFO:teuthology.orchestra.run.smithi019.stdout:Total 51 MB/s | 94 MB 00:01 2023-10-06T13:34:01.263 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:34:01.295 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-10-06T13:34:01.319 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-10-06T13:34:01.377 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:34:01.405 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:34:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2023-10-06T13:34:01.405 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:34:01.406 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:34:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: kernel x86_64 5.14.0-373.el9 CentOS-BaseOS 5.2 M 2023-10-06T13:34:01.406 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2023-10-06T13:34:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-core x86_64 5.14.0-373.el9 CentOS-BaseOS 20 M 2023-10-06T13:34:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules x86_64 5.14.0-373.el9 CentOS-BaseOS 37 M 2023-10-06T13:34:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-core x86_64 5.14.0-373.el9 CentOS-BaseOS 32 M 2023-10-06T13:34:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:34:01.407 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:34:01.408 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:34:01.408 INFO:teuthology.orchestra.run.smithi083.stdout:Install 4 Packages 2023-10-06T13:34:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:34:01.408 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 94 M 2023-10-06T13:34:01.408 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 120 M 2023-10-06T13:34:01.409 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:34:01.679 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:34:01.727 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:34:01.773 INFO:teuthology.orchestra.run.smithi083.stdout:(1/4): kernel-5.14.0-373.el9.x86_64.rpm 14 MB/s | 5.2 MB 00:00 2023-10-06T13:34:02.240 INFO:teuthology.orchestra.run.smithi083.stdout:(2/4): kernel-core-5.14.0-373.el9.x86_64.rpm 23 MB/s | 20 MB 00:00 2023-10-06T13:34:02.751 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:34:02.751 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:34:02.832 INFO:teuthology.orchestra.run.smithi083.stdout:(3/4): kernel-modules-core-5.14.0-373.el9.x86_6 30 MB/s | 32 MB 00:01 2023-10-06T13:34:03.165 INFO:teuthology.orchestra.run.smithi083.stdout:(4/4): kernel-modules-5.14.0-373.el9.x86_64.rpm 21 MB/s | 37 MB 00:01 2023-10-06T13:34:03.169 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:34:03.169 INFO:teuthology.orchestra.run.smithi083.stdout:Total 53 MB/s | 94 MB 00:01 2023-10-06T13:34:03.169 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:34:03.510 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:34:03.510 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:34:04.521 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:34:04.521 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:34:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:34:05.903 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-core-5.14.0-373.el9.x86_64 1/4 2023-10-06T13:34:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-core-5.14.0-373.el9.x86_64 2/4 2023-10-06T13:34:06.913 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:34:07.574 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-modules-core-5.14.0-373.el9.x86_64 1/4 2023-10-06T13:34:07.700 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-core-5.14.0-373.el9.x86_64 2/4 2023-10-06T13:34:08.921 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-373.el9.x86_64 2/4 2023-10-06T13:34:09.123 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-5.14.0-373.el9.x86_64 3/4 2023-10-06T13:34:10.421 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-core-5.14.0-373.el9.x86_64 2/4 2023-10-06T13:34:10.605 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-modules-5.14.0-373.el9.x86_64 3/4 2023-10-06T13:34:14.094 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-373.el9.x86_64 3/4 2023-10-06T13:34:14.188 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-5.14.0-373.el9.x86_64 4/4 2023-10-06T13:34:15.187 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-5.14.0-373.el9.x86_64 3/4 2023-10-06T13:34:15.389 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-5.14.0-373.el9.x86_64 4/4 2023-10-06T13:34:19.192 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-core-5.14.0-373.el9.x86_64 4/4 2023-10-06T13:34:19.995 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-core-5.14.0-373.el9.x86_64 4/4 2023-10-06T13:34:49.359 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-core-5.14.0-373.el9.x86_64 4/4 2023-10-06T13:34:49.401 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-5.14.0-373.el9.x86_64 4/4 2023-10-06T13:34:49.848 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-5.14.0-373.el9.x86_64 4/4 2023-10-06T13:34:49.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-5.14.0-373.el9.x86_64 1/4 2023-10-06T13:34:49.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-core-5.14.0-373.el9.x86_64 2/4 2023-10-06T13:34:49.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-modules-5.14.0-373.el9.x86_64 3/4 2023-10-06T13:34:49.898 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-373.el9.x86_64 4/4 2023-10-06T13:34:49.943 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-373.el9.x86_64 4/4 2023-10-06T13:34:50.237 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-modules-core-5.14.0-373.el9.x86_64 4/4 2023-10-06T13:34:50.238 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:34:50.238 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:34:50.238 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-5.14.0-373.el9.x86_64 2023-10-06T13:34:50.238 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-core-5.14.0-373.el9.x86_64 2023-10-06T13:34:50.238 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-5.14.0-373.el9.x86_64 2023-10-06T13:34:50.238 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-core-5.14.0-373.el9.x86_64 2023-10-06T13:34:50.238 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:34:50.238 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:34:50.519 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-5.14.0-373.el9.x86_64 4/4 2023-10-06T13:34:50.519 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-373.el9.x86_64 1/4 2023-10-06T13:34:50.519 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-core-5.14.0-373.el9.x86_64 2/4 2023-10-06T13:34:50.519 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-5.14.0-373.el9.x86_64 3/4 2023-10-06T13:34:50.758 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T13:34:50.881 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-5.14.0-373.el9.x86_64 2023-10-06T13:34:50.882 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-373.el9.x86_64 2023-10-06T13:34:50.882 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-373.el9.x86_64 2023-10-06T13:34:50.882 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T13:34:50.882 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T13:34:50.883 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-06T13:34:50.883 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi083.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T13:34:50.883 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T13:34:50.944 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-core-5.14.0-373.el9.x86_64 4/4 2023-10-06T13:34:50.944 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:34:50.944 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:34:50.944 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-5.14.0-373.el9.x86_64 2023-10-06T13:34:50.944 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core-5.14.0-373.el9.x86_64 2023-10-06T13:34:50.945 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-5.14.0-373.el9.x86_64 2023-10-06T13:34:50.945 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core-5.14.0-373.el9.x86_64 2023-10-06T13:34:50.945 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:34:50.945 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:34:50.986 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-5.14.0-373.el9.x86_64 2023-10-06T13:34:50.987 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-373.el9.x86_64 2023-10-06T13:34:50.988 DEBUG:teuthology.orchestra.run.smithi083:> sudo rpm -qi grub2-tools 2023-10-06T13:34:51.031 INFO:teuthology.orchestra.run.smithi083.stdout:Name : grub2-tools 2023-10-06T13:34:51.031 INFO:teuthology.orchestra.run.smithi083.stdout:Epoch : 1 2023-10-06T13:34:51.031 INFO:teuthology.orchestra.run.smithi083.stdout:Version : 2.06 2023-10-06T13:34:51.031 INFO:teuthology.orchestra.run.smithi083.stdout:Release : 68.el9 2023-10-06T13:34:51.032 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: x86_64 2023-10-06T13:34:51.032 INFO:teuthology.orchestra.run.smithi083.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-06T13:34:51.032 INFO:teuthology.orchestra.run.smithi083.stdout:Group : Unspecified 2023-10-06T13:34:51.032 INFO:teuthology.orchestra.run.smithi083.stdout:Size : 8264254 2023-10-06T13:34:51.032 INFO:teuthology.orchestra.run.smithi083.stdout:License : GPLv3+ 2023-10-06T13:34:51.032 INFO:teuthology.orchestra.run.smithi083.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-06T13:34:51.032 INFO:teuthology.orchestra.run.smithi083.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-06T13:34:51.032 INFO:teuthology.orchestra.run.smithi083.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-06T13:34:51.032 INFO:teuthology.orchestra.run.smithi083.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-06T13:34:51.032 INFO:teuthology.orchestra.run.smithi083.stdout:Packager : builder@centos.org 2023-10-06T13:34:51.032 INFO:teuthology.orchestra.run.smithi083.stdout:Vendor : CentOS 2023-10-06T13:34:51.032 INFO:teuthology.orchestra.run.smithi083.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T13:34:51.033 INFO:teuthology.orchestra.run.smithi083.stdout:Summary : Support tools for GRUB. 2023-10-06T13:34:51.033 INFO:teuthology.orchestra.run.smithi083.stdout:Description : 2023-10-06T13:34:51.033 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:34:51.033 INFO:teuthology.orchestra.run.smithi083.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T13:34:51.033 INFO:teuthology.orchestra.run.smithi083.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T13:34:51.033 INFO:teuthology.orchestra.run.smithi083.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T13:34:51.033 INFO:teuthology.orchestra.run.smithi083.stdout:hardware devices. 2023-10-06T13:34:51.033 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:34:51.033 INFO:teuthology.orchestra.run.smithi083.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T13:34:51.034 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T13:34:51.034 INFO:teuthology.task.kernel:Updating grub on smithi083 to boot 5.14.0-373.el9.x86_64 2023-10-06T13:34:51.034 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T13:34:51.439 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T13:34:51.675 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2023-10-06T13:34:51.725 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-373.el9.x86_64 2023-10-06T13:34:51.725 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-373.el9.x86_64 2023-10-06T13:34:51.725 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-373.el9.x86_64 2023-10-06T13:34:51.725 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T13:34:51.725 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T13:34:51.726 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-06T13:34:51.726 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T13:34:51.726 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T13:34:51.835 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-373.el9.x86_64 2023-10-06T13:34:51.836 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-373.el9.x86_64 2023-10-06T13:34:51.836 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2023-10-06T13:34:51.874 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2023-10-06T13:34:51.874 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2023-10-06T13:34:51.874 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.06 2023-10-06T13:34:51.874 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 68.el9 2023-10-06T13:34:51.874 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2023-10-06T13:34:51.874 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-06T13:34:51.874 INFO:teuthology.orchestra.run.smithi019.stdout:Group : Unspecified 2023-10-06T13:34:51.874 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 8264254 2023-10-06T13:34:51.874 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2023-10-06T13:34:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-06T13:34:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-06T13:34:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-06T13:34:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-06T13:34:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : builder@centos.org 2023-10-06T13:34:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2023-10-06T13:34:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T13:34:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2023-10-06T13:34:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2023-10-06T13:34:51.875 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:34:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T13:34:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T13:34:51.876 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T13:34:51.876 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2023-10-06T13:34:51.876 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:34:51.876 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T13:34:51.877 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T13:34:51.877 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 5.14.0-373.el9.x86_64 2023-10-06T13:34:51.877 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T13:34:52.541 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2023-10-06T13:34:53.023 INFO:teuthology.orchestra.run.smithi083.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-06T13:34:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:done 2023-10-06T13:34:53.059 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2023-10-06T13:34:53.075 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.fiy2xtVepO 2023-10-06T13:34:53.075 DEBUG:teuthology.orchestra.run.smithi083:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fiy2xtVepO 2023-10-06T13:34:53.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.fiy2xtVepO 2023-10-06T13:34:53.282 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.fiy2xtVepO is 6KB 2023-10-06T13:34:53.329 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.fiy2xtVepO 2023-10-06T13:34:53.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T13:34:53.408 INFO:teuthology.orchestra.run.smithi083.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-06T13:34:53.408 INFO:teuthology.orchestra.run.smithi083.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-06T13:34:53.408 INFO:teuthology.orchestra.run.smithi083.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-06T13:34:53.408 INFO:teuthology.orchestra.run.smithi083.stdout:15949e3c88704300882482eddfd7baae-5.14.0-373.el9.x86_64.conf 2023-10-06T13:34:53.410 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-373.el9.x86_64 2023-10-06T13:34:53.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo shutdown -r now 2023-10-06T13:34:53.946 INFO:teuthology.orchestra.run.smithi019.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-06T13:34:53.987 INFO:teuthology.orchestra.run.smithi019.stderr:done 2023-10-06T13:34:53.989 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2023-10-06T13:34:54.004 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.us3pFwuKyh 2023-10-06T13:34:54.004 DEBUG:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.us3pFwuKyh 2023-10-06T13:34:54.076 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.us3pFwuKyh 2023-10-06T13:34:54.221 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.us3pFwuKyh is 6KB 2023-10-06T13:34:54.270 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.us3pFwuKyh 2023-10-06T13:34:54.286 DEBUG:teuthology.orchestra.run.smithi019:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T13:34:54.354 INFO:teuthology.orchestra.run.smithi019.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-06T13:34:54.354 INFO:teuthology.orchestra.run.smithi019.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-06T13:34:54.354 INFO:teuthology.orchestra.run.smithi019.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-06T13:34:54.354 INFO:teuthology.orchestra.run.smithi019.stdout:15949e3c88704300882482eddfd7baae-5.14.0-373.el9.x86_64.conf 2023-10-06T13:34:54.355 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-373.el9.x86_64 2023-10-06T13:34:54.508 DEBUG:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2023-10-06T13:35:23.560 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-06T13:35:23.561 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2023-10-06T13:35:23.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T13:35:24.512 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-06T13:35:24.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2023-10-06T13:35:24.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T13:35:41.946 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2023-10-06T13:35:50.954 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2023-10-06T13:35:50.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T13:35:54.010 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2023-10-06T13:36:06.022 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2023-10-06T13:36:06.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T13:36:24.541 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T13:36:33.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2023-10-06T13:36:33.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T13:36:34.061 DEBUG:teuthology.orchestra.run.smithi019:> true 2023-10-06T13:36:34.477 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2023-10-06T13:36:34.486 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-373.el9.x86_64"... 2023-10-06T13:36:34.486 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2023-10-06T13:36:34.500 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-373.el9.x86_64 2023-10-06T13:36:34.500 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-373.el9.x86_64 vs 5.14.0-373.el9.x86_64 2023-10-06T13:36:34.501 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T13:36:34.501 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-06T13:36:35.501 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-06T13:36:35.502 DEBUG:teuthology.orchestra.run.smithi019:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T13:36:35.559 INFO:teuthology.orchestra.run.smithi019.stdout:ttyS1 2023-10-06T13:36:35.596 DEBUG:teuthology.parallel:result is None 2023-10-06T13:36:37.876 DEBUG:teuthology.orchestra.run.smithi083:> true 2023-10-06T13:36:38.276 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2023-10-06T13:36:38.307 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-373.el9.x86_64"... 2023-10-06T13:36:38.307 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2023-10-06T13:36:38.320 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-373.el9.x86_64 2023-10-06T13:36:38.321 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-373.el9.x86_64 vs 5.14.0-373.el9.x86_64 2023-10-06T13:36:38.321 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T13:36:38.321 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-06T13:36:39.322 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-06T13:36:39.322 DEBUG:teuthology.orchestra.run.smithi083:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T13:36:39.472 INFO:teuthology.orchestra.run.smithi083.stdout:ttyS1 2023-10-06T13:36:39.505 DEBUG:teuthology.parallel:result is None 2023-10-06T13:36:39.562 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T13:36:39.569 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T13:36:39.570 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T13:36:39.572 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T13:36:39.600 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T13:36:39.615 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T13:36:39.665 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T13:36:39.665 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T13:36:39.668 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T13:36:39.728 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T13:36:39.735 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T13:36:39.735 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T13:36:39.744 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T13:36:39.774 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T13:36:39.790 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T13:36:39.941 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T13:36:39.966 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T13:36:39.968 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T13:36:39.975 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T13:36:39.975 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T13:36:39.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T13:36:40.020 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T13:36:40.033 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T13:36:40.034 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T13:36:40.047 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T13:36:40.085 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T13:36:40.142 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T13:36:40.186 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T13:36:40.265 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T13:36:40.308 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:36:40.309 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T13:36:40.377 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T13:36:40.431 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T13:36:40.479 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T13:36:40.561 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T13:36:40.605 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:36:40.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T13:36:40.672 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2023-10-06T13:36:40.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2023-10-06T13:36:40.730 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T13:36:40.898 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T13:36:41.226 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T13:36:41.271 INFO:teuthology.task.internal:Starting timer... 2023-10-06T13:36:41.272 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T13:36:41.327 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T13:36:41.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2023-10-06T13:36:41.441 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2023-10-06T13:36:41.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2023-10-06T13:36:41.508 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2023-10-06T13:36:41.513 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T13:36:41.513 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2023-10-06T13:36:41.555 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2023-10-06T13:36:41.567 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2023-10-06T13:36:41.598 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2023-10-06T13:36:41.610 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi083.front.sepia.ceph.com': 'permissive'} 2023-10-06T13:36:41.610 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T13:36:41.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T13:36:41.643 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T13:36:41.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T13:36:41.671 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T13:36:41.671 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2023-10-06T13:36:41.724 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2023-10-06T13:36:41.758 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T13:36:41.767 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-06T13:36:41.776 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-06T13:36:41.887 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T13:36:41.903 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T13:36:41.904 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi083.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T13:42:06.567 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi083.front.sepia.ceph.com')] 2023-10-06T13:42:06.568 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2023-10-06T13:42:06.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T13:42:06.663 DEBUG:teuthology.orchestra.run.smithi019:> true 2023-10-06T13:42:06.737 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2023-10-06T13:42:06.737 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2023-10-06T13:42:06.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T13:42:06.822 DEBUG:teuthology.orchestra.run.smithi083:> true 2023-10-06T13:42:06.893 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2023-10-06T13:42:06.893 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T13:42:06.906 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T13:42:06.906 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T13:42:06.906 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T13:42:06.909 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T13:42:06.909 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T13:42:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T13:42:06.961 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T13:42:06.964 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T13:42:06.980 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T13:42:06.998 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2023-10-06T13:42:07.009 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2023-10-06T13:42:07.014 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2023-10-06T13:42:07.022 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2023-10-06T13:42:07.034 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T13:42:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T13:42:07.054 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T13:42:07.054 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T13:42:07.097 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2023-10-06T13:42:07.101 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T13:42:07.101 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2023-10-06T13:42:07.101 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T13:42:07.101 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T13:42:07.101 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T13:42:07.101 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T13:42:07.102 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2023-10-06T13:42:07.104 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T13:42:07.105 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2023-10-06T13:42:07.105 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T13:42:07.105 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T13:42:07.105 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T13:42:07.105 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T13:42:07.106 INFO:teuthology.run_tasks:Running task install... 2023-10-06T13:42:07.117 DEBUG:teuthology.task.install:project ceph 2023-10-06T13:42:07.132 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': '4af54c4a1461e3792e9783e337bb62ce429214f3'}} 2023-10-06T13:42:07.133 DEBUG:teuthology.task.install:config {'debuginfo': True, 'flavor': 'default', 'sha1': '4af54c4a1461e3792e9783e337bb62ce429214f3'} 2023-10-06T13:42:07.133 INFO:teuthology.task.install:Using flavor: default 2023-10-06T13:42:07.138 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2023-10-06T13:42:07.138 INFO:teuthology.task.install:extra packages: [] 2023-10-06T13:42:07.139 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4af54c4a1461e3792e9783e337bb62ce429214f3', 'tag': None, 'wait_for_package': False} 2023-10-06T13:42:07.139 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4af54c4a1461e3792e9783e337bb62ce429214f3 2023-10-06T13:42:07.141 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4af54c4a1461e3792e9783e337bb62ce429214f3', 'tag': None, 'wait_for_package': False} 2023-10-06T13:42:07.141 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4af54c4a1461e3792e9783e337bb62ce429214f3 2023-10-06T13:42:07.284 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-10-04-1350-reef/4af54c4a1461e3792e9783e337bb62ce429214f3/centos/9/flavors/default/ 2023-10-06T13:42:07.284 INFO:teuthology.task.install.rpm:Package version is 18.2.0-596.g4af54c4a 2023-10-06T13:42:07.289 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-10-04-1350-reef/4af54c4a1461e3792e9783e337bb62ce429214f3/centos/9/flavors/default/ 2023-10-06T13:42:07.289 INFO:teuthology.task.install.rpm:Package version is 18.2.0-596.g4af54c4a 2023-10-06T13:42:07.453 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-10-04-1350-reef/4af54c4a1461e3792e9783e337bb62ce429214f3/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-10-04-1350-reef/4af54c4a1461e3792e9783e337bb62ce429214f3/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-10-04-1350-reef/4af54c4a1461e3792e9783e337bb62ce429214f3/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T13:42:07.454 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:42:07.454 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T13:42:07.457 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-10-04-1350-reef/4af54c4a1461e3792e9783e337bb62ce429214f3/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-10-04-1350-reef/4af54c4a1461e3792e9783e337bb62ce429214f3/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-10-04-1350-reef/4af54c4a1461e3792e9783e337bb62ce429214f3/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T13:42:07.457 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:42:07.457 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T13:42:07.485 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2023-10-06T13:42:07.485 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4af54c4a1461e3792e9783e337bb62ce429214f3/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T13:42:07.487 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2023-10-06T13:42:07.487 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4af54c4a1461e3792e9783e337bb62ce429214f3/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T13:42:07.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T13:42:07.558 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T13:42:07.632 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T13:42:07.644 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T13:42:07.674 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2023-10-06T13:42:07.676 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2023-10-06T13:42:07.699 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2023-10-06T13:42:07.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2023-10-06T13:42:07.942 INFO:teuthology.orchestra.run.smithi019.stdout:70 files removed 2023-10-06T13:42:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:70 files removed 2023-10-06T13:42:07.969 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw 2023-10-06T13:42:07.977 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw 2023-10-06T13:42:08.762 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 162 kB/s | 77 kB 00:00 2023-10-06T13:42:08.777 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 163 kB/s | 77 kB 00:00 2023-10-06T13:42:09.121 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-10-06T13:42:09.145 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-10-06T13:42:09.430 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 6.8 kB/s | 1.9 kB 00:00 2023-10-06T13:42:09.504 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2023-10-06T13:42:10.006 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-9 - AppStream 14 MB/s | 6.3 MB 00:00 2023-10-06T13:42:10.066 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS-9 - AppStream 10 MB/s | 6.3 MB 00:00 2023-10-06T13:42:11.492 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2023-10-06T13:42:11.498 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2023-10-06T13:42:13.157 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 7.9 MB 00:01 2023-10-06T13:42:13.389 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 7.9 MB 00:01 2023-10-06T13:42:15.017 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 21 MB/s | 18 MB 00:00 2023-10-06T13:42:16.124 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 11 MB/s | 18 MB 00:01 2023-10-06T13:42:19.876 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 5.7 MB 00:01 2023-10-06T13:42:21.142 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 5.7 MB 00:01 2023-10-06T13:42:21.419 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 14 kB 00:00 2023-10-06T13:42:22.584 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 14 kB 00:00 2023-10-06T13:42:23.014 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 13 MB/s | 19 MB 00:01 2023-10-06T13:42:23.544 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 22 MB/s | 19 MB 00:00 2023-10-06T13:42:27.429 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2023-10-06T13:42:28.178 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2023-10-06T13:42:29.896 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:42:29.910 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:42:29.910 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2023-10-06T13:42:29.910 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:42:29.910 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:42:29.910 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 7.7 M 2023-10-06T13:42:29.910 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2023-10-06T13:42:29.910 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 3.3 M 2023-10-06T13:42:29.911 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 3.0 M 2023-10-06T13:42:29.911 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2023-10-06T13:42:29.911 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-06T13:42:29.911 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 5.2 M 2023-10-06T13:42:29.911 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 18 M 2023-10-06T13:42:29.911 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 24 k 2023-10-06T13:42:29.911 INFO:teuthology.orchestra.run.smithi083.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-06T13:42:29.911 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-06T13:42:29.911 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-06T13:42:29.911 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-06T13:42:29.911 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 653 k 2023-10-06T13:42:29.912 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-06T13:42:29.912 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-06T13:42:29.912 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-06T13:42:29.912 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 474 k 2023-10-06T13:42:29.912 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-06T13:42:29.912 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 4.4 M 2023-10-06T13:42:29.912 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-06T13:42:29.912 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-06T13:42:29.912 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-06T13:42:29.912 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 45 k 2023-10-06T13:42:29.912 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 124 k 2023-10-06T13:42:29.913 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 156 k 2023-10-06T13:42:29.913 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-06T13:42:29.913 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 320 k 2023-10-06T13:42:29.913 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 297 k 2023-10-06T13:42:29.913 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 99 k 2023-10-06T13:42:29.913 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-06T13:42:29.913 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-06T13:42:29.913 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:42:29.913 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:42:29.913 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:42:29.913 INFO:teuthology.orchestra.run.smithi083.stdout:Install 28 Packages 2023-10-06T13:42:29.913 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 2 Packages 2023-10-06T13:42:29.914 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:42:29.914 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 53 M 2023-10-06T13:42:29.914 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:42:30.522 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:42:30.532 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:42:30.532 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-10-06T13:42:30.532 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:42:30.533 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:42:30.533 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 7.7 M 2023-10-06T13:42:30.533 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2023-10-06T13:42:30.533 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 3.3 M 2023-10-06T13:42:30.533 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 3.0 M 2023-10-06T13:42:30.533 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-10-06T13:42:30.533 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-06T13:42:30.533 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 5.2 M 2023-10-06T13:42:30.533 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 18 M 2023-10-06T13:42:30.533 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 24 k 2023-10-06T13:42:30.533 INFO:teuthology.orchestra.run.smithi019.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-06T13:42:30.533 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-06T13:42:30.534 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-06T13:42:30.534 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-06T13:42:30.534 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 653 k 2023-10-06T13:42:30.534 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-06T13:42:30.534 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-06T13:42:30.534 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-06T13:42:30.534 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 474 k 2023-10-06T13:42:30.534 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-06T13:42:30.534 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 4.4 M 2023-10-06T13:42:30.534 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-06T13:42:30.534 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-06T13:42:30.534 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-06T13:42:30.535 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 45 k 2023-10-06T13:42:30.535 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 124 k 2023-10-06T13:42:30.535 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 156 k 2023-10-06T13:42:30.535 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-06T13:42:30.535 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 320 k 2023-10-06T13:42:30.535 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 297 k 2023-10-06T13:42:30.535 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 99 k 2023-10-06T13:42:30.535 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-06T13:42:30.535 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-06T13:42:30.535 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:42:30.535 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:42:30.536 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:42:30.536 INFO:teuthology.orchestra.run.smithi019.stdout:Install 28 Packages 2023-10-06T13:42:30.536 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 2 Packages 2023-10-06T13:42:30.536 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:42:30.536 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 53 M 2023-10-06T13:42:30.536 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:42:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:(1/30): ceph-base-18.2.0-596.g4af54c4a.el9.x86_ 6.7 MB/s | 5.2 MB 00:00 2023-10-06T13:42:30.897 INFO:teuthology.orchestra.run.smithi083.stdout:(2/30): ceph-selinux-18.2.0-596.g4af54c4a.el9.x 321 kB/s | 24 kB 00:00 2023-10-06T13:42:31.005 INFO:teuthology.orchestra.run.smithi083.stdout:(3/30): libcephfs2-18.2.0-596.g4af54c4a.el9.x86 5.9 MB/s | 653 kB 00:00 2023-10-06T13:42:31.114 INFO:teuthology.orchestra.run.smithi083.stdout:(4/30): libradosstriper1-18.2.0-596.g4af54c4a.e 4.3 MB/s | 474 kB 00:00 2023-10-06T13:42:31.340 INFO:teuthology.orchestra.run.smithi083.stdout:(5/30): ceph-common-18.2.0-596.g4af54c4a.el9.x8 14 MB/s | 18 MB 00:01 2023-10-06T13:42:31.406 INFO:teuthology.orchestra.run.smithi083.stdout:(6/30): python3-ceph-argparse-18.2.0-596.g4af54 680 kB/s | 45 kB 00:00 2023-10-06T13:42:31.533 INFO:teuthology.orchestra.run.smithi083.stdout:(7/30): librgw2-18.2.0-596.g4af54c4a.el9.x86_64 11 MB/s | 4.4 MB 00:00 2023-10-06T13:42:31.557 INFO:teuthology.orchestra.run.smithi083.stdout:(8/30): python3-ceph-common-18.2.0-596.g4af54c4 828 kB/s | 124 kB 00:00 2023-10-06T13:42:31.582 INFO:teuthology.orchestra.run.smithi083.stdout:(9/30): python3-cephfs-18.2.0-596.g4af54c4a.el9 3.0 MB/s | 156 kB 00:00 2023-10-06T13:42:31.624 INFO:teuthology.orchestra.run.smithi083.stdout:(10/30): python3-rados-18.2.0-596.g4af54c4a.el9 4.7 MB/s | 320 kB 00:00 2023-10-06T13:42:31.658 INFO:teuthology.orchestra.run.smithi083.stdout:(11/30): python3-rbd-18.2.0-596.g4af54c4a.el9.x 3.9 MB/s | 297 kB 00:00 2023-10-06T13:42:31.691 INFO:teuthology.orchestra.run.smithi083.stdout:(12/30): python3-rgw-18.2.0-596.g4af54c4a.el9.x 1.4 MB/s | 99 kB 00:00 2023-10-06T13:42:31.808 INFO:teuthology.orchestra.run.smithi083.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 704 kB/s | 106 kB 00:00 2023-10-06T13:42:31.850 INFO:teuthology.orchestra.run.smithi083.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 159 kB 00:00 2023-10-06T13:42:31.893 INFO:teuthology.orchestra.run.smithi083.stdout:(15/30): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2023-10-06T13:42:31.917 INFO:teuthology.orchestra.run.smithi083.stdout:(16/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.1 MB/s | 282 kB 00:00 2023-10-06T13:42:31.943 INFO:teuthology.orchestra.run.smithi083.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 827 kB/s | 41 kB 00:00 2023-10-06T13:42:31.962 INFO:teuthology.orchestra.run.smithi019.stdout:(1/30): ceph-base-18.2.0-596.g4af54c4a.el9.x86_ 4.8 MB/s | 5.2 MB 00:01 2023-10-06T13:42:32.004 INFO:teuthology.orchestra.run.smithi019.stdout:(2/30): ceph-selinux-18.2.0-596.g4af54c4a.el9.x 581 kB/s | 24 kB 00:00 2023-10-06T13:42:32.096 INFO:teuthology.orchestra.run.smithi019.stdout:(3/30): libcephfs2-18.2.0-596.g4af54c4a.el9.x86 7.0 MB/s | 653 kB 00:00 2023-10-06T13:42:32.205 INFO:teuthology.orchestra.run.smithi019.stdout:(4/30): libradosstriper1-18.2.0-596.g4af54c4a.e 4.3 MB/s | 474 kB 00:00 2023-10-06T13:42:32.326 INFO:teuthology.orchestra.run.smithi083.stdout:(18/30): ceph-radosgw-18.2.0-596.g4af54c4a.el9. 3.4 MB/s | 7.7 MB 00:02 2023-10-06T13:42:32.394 INFO:teuthology.orchestra.run.smithi083.stdout:(19/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 95 kB/s | 45 kB 00:00 2023-10-06T13:42:32.577 INFO:teuthology.orchestra.run.smithi083.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 175 kB/s | 111 kB 00:00 2023-10-06T13:42:32.671 INFO:teuthology.orchestra.run.smithi019.stdout:(5/30): ceph-radosgw-18.2.0-596.g4af54c4a.el9.x 4.3 MB/s | 7.7 MB 00:01 2023-10-06T13:42:32.711 INFO:teuthology.orchestra.run.smithi083.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 805 kB/s | 309 kB 00:00 2023-10-06T13:42:32.730 INFO:teuthology.orchestra.run.smithi019.stdout:(6/30): librgw2-18.2.0-596.g4af54c4a.el9.x86_64 8.4 MB/s | 4.4 MB 00:00 2023-10-06T13:42:32.755 INFO:teuthology.orchestra.run.smithi019.stdout:(7/30): python3-ceph-argparse-18.2.0-596.g4af54 539 kB/s | 45 kB 00:00 2023-10-06T13:42:32.781 INFO:teuthology.orchestra.run.smithi019.stdout:(8/30): python3-ceph-common-18.2.0-596.g4af54c4 2.4 MB/s | 124 kB 00:00 2023-10-06T13:42:32.814 INFO:teuthology.orchestra.run.smithi019.stdout:(9/30): python3-cephfs-18.2.0-596.g4af54c4a.el9 2.6 MB/s | 156 kB 00:00 2023-10-06T13:42:32.836 INFO:teuthology.orchestra.run.smithi083.stdout:(22/30): libarrow-doc-9.0.0-9.el9.noarch.rpm 94 kB/s | 24 kB 00:00 2023-10-06T13:42:32.839 INFO:teuthology.orchestra.run.smithi019.stdout:(10/30): python3-rados-18.2.0-596.g4af54c4a.el9 5.3 MB/s | 320 kB 00:00 2023-10-06T13:42:32.881 INFO:teuthology.orchestra.run.smithi019.stdout:(11/30): python3-rbd-18.2.0-596.g4af54c4a.el9.x 4.3 MB/s | 297 kB 00:00 2023-10-06T13:42:32.907 INFO:teuthology.orchestra.run.smithi019.stdout:(12/30): python3-rgw-18.2.0-596.g4af54c4a.el9.x 1.4 MB/s | 99 kB 00:00 2023-10-06T13:42:32.995 INFO:teuthology.orchestra.run.smithi083.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 171 kB/s | 49 kB 00:00 2023-10-06T13:42:33.024 INFO:teuthology.orchestra.run.smithi019.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 745 kB/s | 106 kB 00:00 2023-10-06T13:42:33.049 INFO:teuthology.orchestra.run.smithi019.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 159 kB 00:00 2023-10-06T13:42:33.062 INFO:teuthology.orchestra.run.smithi083.stdout:(24/30): libarrow-9.0.0-9.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2023-10-06T13:42:33.087 INFO:teuthology.orchestra.run.smithi083.stdout:(25/30): libunwind-1.6.2-1.el9.x86_64.rpm 269 kB/s | 67 kB 00:00 2023-10-06T13:42:33.099 INFO:teuthology.orchestra.run.smithi019.stdout:(15/30): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2023-10-06T13:42:33.120 INFO:teuthology.orchestra.run.smithi083.stdout:(26/30): parquet-libs-9.0.0-9.el9.x86_64.rpm 6.7 MB/s | 837 kB 00:00 2023-10-06T13:42:33.125 INFO:teuthology.orchestra.run.smithi019.stdout:(16/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2023-10-06T13:42:33.150 INFO:teuthology.orchestra.run.smithi019.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 826 kB/s | 41 kB 00:00 2023-10-06T13:42:33.196 INFO:teuthology.orchestra.run.smithi083.stdout:(27/30): re2-20211101-3.el9.x86_64.rpm 1.8 MB/s | 192 kB 00:00 2023-10-06T13:42:33.238 INFO:teuthology.orchestra.run.smithi083.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2023-10-06T13:42:33.325 INFO:teuthology.orchestra.run.smithi019.stdout:(18/30): ceph-common-18.2.0-596.g4af54c4a.el9.x 7.5 MB/s | 18 MB 00:02 2023-10-06T13:42:33.505 INFO:teuthology.orchestra.run.smithi083.stdout:(29/30): librados2-18.2.0-596.g4af54c4a.el9.x86 8.5 MB/s | 3.3 MB 00:00 2023-10-06T13:42:33.563 INFO:teuthology.orchestra.run.smithi083.stdout:(30/30): librbd1-18.2.0-596.g4af54c4a.el9.x86_6 8.2 MB/s | 3.0 MB 00:00 2023-10-06T13:42:33.569 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:42:33.570 INFO:teuthology.orchestra.run.smithi083.stdout:Total 14 MB/s | 53 MB 00:03 2023-10-06T13:42:33.582 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:42:33.592 INFO:teuthology.orchestra.run.smithi019.stdout:(19/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 97 kB/s | 45 kB 00:00 2023-10-06T13:42:33.619 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:42:33.619 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:42:33.876 INFO:teuthology.orchestra.run.smithi019.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 153 kB/s | 111 kB 00:00 2023-10-06T13:42:34.034 INFO:teuthology.orchestra.run.smithi019.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 436 kB/s | 309 kB 00:00 2023-10-06T13:42:34.118 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:42:34.118 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:42:34.135 INFO:teuthology.orchestra.run.smithi019.stdout:(22/30): libarrow-doc-9.0.0-9.el9.noarch.rpm 94 kB/s | 24 kB 00:00 2023-10-06T13:42:34.277 INFO:teuthology.orchestra.run.smithi019.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 201 kB/s | 49 kB 00:00 2023-10-06T13:42:34.402 INFO:teuthology.orchestra.run.smithi019.stdout:(24/30): libarrow-9.0.0-9.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2023-10-06T13:42:34.427 INFO:teuthology.orchestra.run.smithi019.stdout:(25/30): libunwind-1.6.2-1.el9.x86_64.rpm 231 kB/s | 67 kB 00:00 2023-10-06T13:42:34.461 INFO:teuthology.orchestra.run.smithi019.stdout:(26/30): parquet-libs-9.0.0-9.el9.x86_64.rpm 4.5 MB/s | 837 kB 00:00 2023-10-06T13:42:34.527 INFO:teuthology.orchestra.run.smithi019.stdout:(27/30): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2023-10-06T13:42:34.570 INFO:teuthology.orchestra.run.smithi019.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2023-10-06T13:42:34.778 INFO:teuthology.orchestra.run.smithi019.stdout:(29/30): librados2-18.2.0-596.g4af54c4a.el9.x86 10 MB/s | 3.3 MB 00:00 2023-10-06T13:42:34.945 INFO:teuthology.orchestra.run.smithi019.stdout:(30/30): librbd1-18.2.0-596.g4af54c4a.el9.x86_6 7.2 MB/s | 3.0 MB 00:00 2023-10-06T13:42:34.951 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:42:34.951 INFO:teuthology.orchestra.run.smithi019.stdout:Total 12 MB/s | 53 MB 00:04 2023-10-06T13:42:34.963 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:42:35.005 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:42:35.006 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:42:35.496 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:42:35.539 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:42:35.539 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:42:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2023-10-06T13:42:35.607 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/32 2023-10-06T13:42:35.660 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/32 2023-10-06T13:42:35.975 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2023-10-06T13:42:36.034 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:18.2.0-596.g4af54c4a.el9.x86_64 5/32 2023-10-06T13:42:36.125 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:18.2.0-596.g4af54c4a.el9.x86_64 5/32 2023-10-06T13:42:36.219 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:18.2.0-596.g4af54c4a.el9.x86_64 6/32 2023-10-06T13:42:36.243 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:18.2.0-596.g4af54c4a.el9.x86_64 7/32 2023-10-06T13:42:36.306 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:18.2.0-596.g4af54c4a.el9.x86_64 7/32 2023-10-06T13:42:36.343 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2023-10-06T13:42:36.386 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:18.2.0-596.g4af54c4a.el9.x 9/32 2023-10-06T13:42:36.456 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:18.2.0-596.g4af54c4a.el9.x86_64 10/32 2023-10-06T13:42:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:18.2.0-596.g4af54c4a.el9.x86_64 11/32 2023-10-06T13:42:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:18.2.0-596.g4af54c4a.el9.x86_64 11/32 2023-10-06T13:42:36.633 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2023-10-06T13:42:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 13/32 2023-10-06T13:42:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2023-10-06T13:42:36.809 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2023-10-06T13:42:36.809 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 16/32 2023-10-06T13:42:36.822 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2023-10-06T13:42:36.855 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-10-06T13:42:37.102 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:42:37.401 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2023-10-06T13:42:37.401 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/32 2023-10-06T13:42:37.401 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/32 2023-10-06T13:42:37.401 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2023-10-06T13:42:37.401 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:18.2.0-596.g4af54c4a.el9.x86_64 20/32 2023-10-06T13:42:37.401 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:18.2.0-596.g4af54c4a.el9.x86_64 20/32 2023-10-06T13:42:37.402 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:18.2.0-596.g4af54c4a.el9.x86_64 21/32 2023-10-06T13:42:37.402 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2023-10-06T13:42:37.613 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2023-10-06T13:42:37.666 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:18.2.0-596.g4af54c4a.el9.x86_64 5/32 2023-10-06T13:42:37.834 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:18.2.0-596.g4af54c4a.el9.x86_64 5/32 2023-10-06T13:42:37.834 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:18.2.0-596.g4af54c4a.el9.x86_64 6/32 2023-10-06T13:42:37.834 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 23/32 2023-10-06T13:42:37.834 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:18.2.0-596.g4af54c4a.el9.x86_64 24/32 2023-10-06T13:42:37.835 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:18.2.0-596.g4af54c4a.el9.x86_64 24/32 2023-10-06T13:42:37.850 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:18.2.0-596.g4af54c4a.el9.x86_64 7/32 2023-10-06T13:42:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:18.2.0-596.g4af54c4a.el9.x86_64 7/32 2023-10-06T13:42:38.017 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:18.2.0-596.g4af54c4a.el9.x86_64 25/32 2023-10-06T13:42:38.017 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:18.2.0-596.g4af54c4a.el9.x86 26/32 2023-10-06T13:42:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2023-10-06T13:42:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:18.2.0-596.g4af54c4a.el9.x 9/32 2023-10-06T13:42:38.054 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:18.2.0-596.g4af54c4a.el9.x86_64 10/32 2023-10-06T13:42:38.079 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:18.2.0-596.g4af54c4a.el9.x86_64 11/32 2023-10-06T13:42:38.129 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:18.2.0-596.g4af54c4a.el9.x86_64 11/32 2023-10-06T13:42:38.237 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2023-10-06T13:42:38.269 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 13/32 2023-10-06T13:42:38.317 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2023-10-06T13:42:38.350 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2023-10-06T13:42:38.562 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 16/32 2023-10-06T13:42:38.562 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2023-10-06T13:42:38.562 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-10-06T13:42:38.762 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2023-10-06T13:42:38.825 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:18.2.0-596.g4af54c4a.el9.x86_64 20/32 2023-10-06T13:42:38.925 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:18.2.0-596.g4af54c4a.el9.x86_64 20/32 2023-10-06T13:42:38.955 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:18.2.0-596.g4af54c4a.el9.x86_64 21/32 2023-10-06T13:42:38.992 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2023-10-06T13:42:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 23/32 2023-10-06T13:42:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:18.2.0-596.g4af54c4a.el9.x86_64 24/32 2023-10-06T13:42:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:18.2.0-596.g4af54c4a.el9.x86_64 24/32 2023-10-06T13:42:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:18.2.0-596.g4af54c4a.el9.x86_64 25/32 2023-10-06T13:42:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:18.2.0-596.g4af54c4a.el9.x86 26/32 2023-10-06T13:42:40.017 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:18.2.0-596.g4af54c4a.el9.x86_64 27/32 2023-10-06T13:42:40.069 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:18.2.0-596.g4af54c4a.el9.x86_64 27/32 2023-10-06T13:42:40.639 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:18.2.0-596.g4af54c4a.el9.x86_64 27/32 2023-10-06T13:42:40.681 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:18.2.0-596.g4af54c4a.el9.x86_64 28/32 2023-10-06T13:42:40.764 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:18.2.0-596.g4af54c4a.el9.x86_64 28/32 2023-10-06T13:42:40.764 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-06T13:42:40.764 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-06T13:42:40.764 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:42:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:18.2.0-596.g4af54c4a.el9.x86_64 29/32 2023-10-06T13:42:41.766 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:18.2.0-596.g4af54c4a.el9.x86_64 27/32 2023-10-06T13:42:41.802 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:18.2.0-596.g4af54c4a.el9.x86_64 27/32 2023-10-06T13:42:42.381 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:18.2.0-596.g4af54c4a.el9.x86_64 27/32 2023-10-06T13:42:42.431 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:18.2.0-596.g4af54c4a.el9.x86_64 28/32 2023-10-06T13:42:42.522 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:18.2.0-596.g4af54c4a.el9.x86_64 28/32 2023-10-06T13:42:42.523 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-06T13:42:42.523 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-06T13:42:42.523 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:42:42.607 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:18.2.0-596.g4af54c4a.el9.x86_64 29/32 2023-10-06T13:42:50.423 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:18.2.0-596.g4af54c4a.el9.x86_64 29/32 2023-10-06T13:42:50.423 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2023-10-06T13:42:50.423 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2023-10-06T13:42:50.424 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2023-10-06T13:42:50.424 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2023-10-06T13:42:50.424 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2023-10-06T13:42:50.424 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2023-10-06T13:42:50.424 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2023-10-06T13:42:50.424 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:42:51.163 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:18.2.0-596.g4af54c4a.el9.x86_64 30/32 2023-10-06T13:42:51.187 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-596.g4af54c4a.el9.x86_64 30/32 2023-10-06T13:42:51.189 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:42:51.189 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-06T13:42:51.189 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-06T13:42:51.189 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-06T13:42:51.189 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:42:51.190 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-10-06T13:42:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-10-06T13:42:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2023-10-06T13:42:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2023-10-06T13:42:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:18.2.0-596.g4af54c4a.el9.x86_64 1/32 2023-10-06T13:42:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:18.2.0-596.g4af54c4a.el9.x86_64 2/32 2023-10-06T13:42:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:18.2.0-596.g4af54c4a.el9.x86_64 3/32 2023-10-06T13:42:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:18.2.0-596.g4af54c4a.el9.x86_64 4/32 2023-10-06T13:42:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:18.2.0-596.g4af54c4a.el9.x86_64 5/32 2023-10-06T13:42:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:18.2.0-596.g4af54c4a.el9.x86_64 6/32 2023-10-06T13:42:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:18.2.0-596.g4af54c4a.el9.x86_64 7/32 2023-10-06T13:42:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:18.2.0-596.g4af54c4a.el9.x 8/32 2023-10-06T13:42:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:18.2.0-596.g4af54c4a.el9.x86 9/32 2023-10-06T13:42:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:18.2.0-596.g4af54c4a.el9.x86_64 10/32 2023-10-06T13:42:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:18.2.0-596.g4af54c4a.el9.x86_64 11/32 2023-10-06T13:42:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:18.2.0-596.g4af54c4a.el9.x86_64 12/32 2023-10-06T13:42:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:18.2.0-596.g4af54c4a.el9.x86_64 13/32 2023-10-06T13:42:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2023-10-06T13:42:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2023-10-06T13:42:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/32 2023-10-06T13:42:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/32 2023-10-06T13:42:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-10-06T13:42:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/32 2023-10-06T13:42:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2023-10-06T13:42:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2023-10-06T13:42:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 22/32 2023-10-06T13:42:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 23/32 2023-10-06T13:42:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2023-10-06T13:42:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2023-10-06T13:42:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 26/32 2023-10-06T13:42:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2023-10-06T13:42:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2023-10-06T13:42:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:18.2.0-596.g4af54c4a.el9.x86_64 29/32 2023-10-06T13:42:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2023-10-06T13:42:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:18.2.0-596.g4af54c4a.el9.x86_64 31/32 2023-10-06T13:42:52.761 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:18.2.0-596.g4af54c4a.el9.x86_64 29/32 2023-10-06T13:42:52.762 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2023-10-06T13:42:52.762 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2023-10-06T13:42:52.762 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2023-10-06T13:42:52.762 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2023-10-06T13:42:52.762 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2023-10-06T13:42:52.762 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2023-10-06T13:42:52.762 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2023-10-06T13:42:52.762 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:42:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2023-10-06T13:42:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:42:53.501 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2023-10-06T13:42:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.501 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:42:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-06T13:42:53.501 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-06T13:42:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-06T13:42:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-06T13:42:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-06T13:42:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-06T13:42:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-06T13:42:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-06T13:42:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-06T13:42:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-06T13:42:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-06T13:42:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-06T13:42:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.503 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-06T13:42:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-3.el9.x86_64 2023-10-06T13:42:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-06T13:42:53.504 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:42:53.504 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:42:53.519 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:18.2.0-596.g4af54c4a.el9.x86_64 30/32 2023-10-06T13:42:53.551 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-596.g4af54c4a.el9.x86_64 30/32 2023-10-06T13:42:53.551 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:42:53.551 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-06T13:42:53.551 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-06T13:42:53.551 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-06T13:42:53.551 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:42:53.553 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-10-06T13:42:53.622 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-10-06T13:42:53.622 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2023-10-06T13:42:53.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-test 2023-10-06T13:42:54.454 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:27 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:42:54.591 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:42:54.592 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:42:54.592 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2023-10-06T13:42:54.592 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:42:54.592 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:42:54.592 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 40 M 2023-10-06T13:42:54.592 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2023-10-06T13:42:54.592 INFO:teuthology.orchestra.run.smithi083.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-06T13:42:54.592 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 161 k 2023-10-06T13:42:54.593 INFO:teuthology.orchestra.run.smithi083.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-06T13:42:54.593 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-06T13:42:54.593 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-06T13:42:54.593 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:42:54.593 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:42:54.593 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:42:54.593 INFO:teuthology.orchestra.run.smithi083.stdout:Install 6 Packages 2023-10-06T13:42:54.593 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:42:54.594 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 40 M 2023-10-06T13:42:54.594 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 167 M 2023-10-06T13:42:54.594 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:42:54.714 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2023-10-06T13:42:54.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:18.2.0-596.g4af54c4a.el9.x86_64 1/32 2023-10-06T13:42:54.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:18.2.0-596.g4af54c4a.el9.x86_64 2/32 2023-10-06T13:42:54.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:18.2.0-596.g4af54c4a.el9.x86_64 3/32 2023-10-06T13:42:54.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:18.2.0-596.g4af54c4a.el9.x86_64 4/32 2023-10-06T13:42:54.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:18.2.0-596.g4af54c4a.el9.x86_64 5/32 2023-10-06T13:42:54.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:18.2.0-596.g4af54c4a.el9.x86_64 6/32 2023-10-06T13:42:54.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:18.2.0-596.g4af54c4a.el9.x86_64 7/32 2023-10-06T13:42:54.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:18.2.0-596.g4af54c4a.el9.x 8/32 2023-10-06T13:42:54.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:18.2.0-596.g4af54c4a.el9.x86 9/32 2023-10-06T13:42:54.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:18.2.0-596.g4af54c4a.el9.x86_64 10/32 2023-10-06T13:42:54.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:18.2.0-596.g4af54c4a.el9.x86_64 11/32 2023-10-06T13:42:54.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:18.2.0-596.g4af54c4a.el9.x86_64 12/32 2023-10-06T13:42:54.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:18.2.0-596.g4af54c4a.el9.x86_64 13/32 2023-10-06T13:42:54.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2023-10-06T13:42:54.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2023-10-06T13:42:54.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/32 2023-10-06T13:42:54.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/32 2023-10-06T13:42:54.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-10-06T13:42:54.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/32 2023-10-06T13:42:54.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2023-10-06T13:42:54.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2023-10-06T13:42:54.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 22/32 2023-10-06T13:42:54.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 23/32 2023-10-06T13:42:54.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2023-10-06T13:42:54.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2023-10-06T13:42:54.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 26/32 2023-10-06T13:42:54.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2023-10-06T13:42:54.717 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2023-10-06T13:42:54.717 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:18.2.0-596.g4af54c4a.el9.x86_64 29/32 2023-10-06T13:42:54.717 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2023-10-06T13:42:54.717 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:18.2.0-596.g4af54c4a.el9.x86_64 31/32 2023-10-06T13:42:54.868 INFO:teuthology.orchestra.run.smithi083.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.4 MB/s | 217 kB 00:00 2023-10-06T13:42:54.927 INFO:teuthology.orchestra.run.smithi083.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 300 kB 00:00 2023-10-06T13:42:54.960 INFO:teuthology.orchestra.run.smithi083.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2023-10-06T13:42:55.011 INFO:teuthology.orchestra.run.smithi083.stdout:(4/6): libcephsqlite-18.2.0-596.g4af54c4a.el9.x 543 kB/s | 161 kB 00:00 2023-10-06T13:42:55.186 INFO:teuthology.orchestra.run.smithi083.stdout:(5/6): jq-1.6-15.el9.x86_64.rpm 829 kB/s | 187 kB 00:00 2023-10-06T13:42:55.690 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2023-10-06T13:42:55.690 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:42:55.690 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2023-10-06T13:42:55.691 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.691 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.691 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:42:55.691 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-06T13:42:55.691 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.691 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.691 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.691 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.691 INFO:teuthology.orchestra.run.smithi019.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-06T13:42:55.692 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-06T13:42:55.692 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-06T13:42:55.692 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-06T13:42:55.692 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.692 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-06T13:42:55.692 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-06T13:42:55.692 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-06T13:42:55.692 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.693 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-06T13:42:55.693 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.693 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-06T13:42:55.693 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-06T13:42:55.693 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-06T13:42:55.693 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.693 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.693 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.693 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-06T13:42:55.694 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.694 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.694 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:42:55.694 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-3.el9.x86_64 2023-10-06T13:42:55.694 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-06T13:42:55.694 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:42:55.694 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:42:56.035 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-test 2023-10-06T13:42:56.652 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:28 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:42:56.803 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:42:56.804 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:42:56.804 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-10-06T13:42:56.804 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:42:56.804 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:42:56.804 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 40 M 2023-10-06T13:42:56.804 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-10-06T13:42:56.804 INFO:teuthology.orchestra.run.smithi019.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-06T13:42:56.805 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 161 k 2023-10-06T13:42:56.805 INFO:teuthology.orchestra.run.smithi019.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-06T13:42:56.805 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-06T13:42:56.805 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-06T13:42:56.805 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:42:56.805 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:42:56.805 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:42:56.805 INFO:teuthology.orchestra.run.smithi019.stdout:Install 6 Packages 2023-10-06T13:42:56.805 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:42:56.806 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 40 M 2023-10-06T13:42:56.806 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 167 M 2023-10-06T13:42:56.806 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:42:56.911 INFO:teuthology.orchestra.run.smithi083.stdout:(6/6): ceph-test-18.2.0-596.g4af54c4a.el9.x86_6 18 MB/s | 40 MB 00:02 2023-10-06T13:42:56.914 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:42:56.915 INFO:teuthology.orchestra.run.smithi083.stdout:Total 17 MB/s | 40 MB 00:02 2023-10-06T13:42:56.928 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:42:56.949 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:42:56.949 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:42:57.111 INFO:teuthology.orchestra.run.smithi019.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 915 kB/s | 217 kB 00:00 2023-10-06T13:42:57.161 INFO:teuthology.orchestra.run.smithi019.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 300 kB 00:00 2023-10-06T13:42:57.186 INFO:teuthology.orchestra.run.smithi019.stdout:(3/6): libcephsqlite-18.2.0-596.g4af54c4a.el9.x 516 kB/s | 161 kB 00:00 2023-10-06T13:42:57.211 INFO:teuthology.orchestra.run.smithi019.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2023-10-06T13:42:57.287 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:42:57.287 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:42:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:42:57.895 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2023-10-06T13:42:57.937 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2023-10-06T13:42:57.978 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-10-06T13:42:58.020 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : jq-1.6-15.el9.x86_64 4/6 2023-10-06T13:42:58.048 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:18.2.0-596.g4af54c4a.el9.x86_64 5/6 2023-10-06T13:42:58.070 INFO:teuthology.orchestra.run.smithi019.stdout:(5/6): jq-1.6-15.el9.x86_64.rpm 211 kB/s | 187 kB 00:00 2023-10-06T13:42:59.129 INFO:teuthology.orchestra.run.smithi019.stdout:(6/6): ceph-test-18.2.0-596.g4af54c4a.el9.x86_6 18 MB/s | 40 MB 00:02 2023-10-06T13:42:59.132 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:42:59.132 INFO:teuthology.orchestra.run.smithi019.stdout:Total 17 MB/s | 40 MB 00:02 2023-10-06T13:42:59.146 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:42:59.169 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:42:59.169 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:42:59.525 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:42:59.525 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:43:00.087 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:43:00.162 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2023-10-06T13:43:00.196 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2023-10-06T13:43:00.234 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-10-06T13:43:00.280 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : jq-1.6-15.el9.x86_64 4/6 2023-10-06T13:43:00.306 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:18.2.0-596.g4af54c4a.el9.x86_64 5/6 2023-10-06T13:43:01.749 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:18.2.0-596.g4af54c4a.el9.x86_64 5/6 2023-10-06T13:43:01.822 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:18.2.0-596.g4af54c4a.el9.x86_64 6/6 2023-10-06T13:43:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-test-2:18.2.0-596.g4af54c4a.el9.x86_64 6/6 2023-10-06T13:43:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:18.2.0-596.g4af54c4a.el9.x86_64 1/6 2023-10-06T13:43:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:18.2.0-596.g4af54c4a.el9.x86_64 2/6 2023-10-06T13:43:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-10-06T13:43:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2023-10-06T13:43:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2023-10-06T13:43:04.252 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:18.2.0-596.g4af54c4a.el9.x86_64 5/6 2023-10-06T13:43:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : jq-1.6-15.el9.x86_64 6/6 2023-10-06T13:43:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:04.290 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:43:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: jq-1.6-15.el9.x86_64 2023-10-06T13:43:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-06T13:43:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-06T13:43:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-06T13:43:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:04.291 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:04.334 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:18.2.0-596.g4af54c4a.el9.x86_64 6/6 2023-10-06T13:43:04.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph 2023-10-06T13:43:05.183 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:38 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================== 2023-10-06T13:43:05.333 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2023-10-06T13:43:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================== 2023-10-06T13:43:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:43:05.333 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 6.5 k 2023-10-06T13:43:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2023-10-06T13:43:05.334 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 2.1 M 2023-10-06T13:43:05.334 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 1.4 M 2023-10-06T13:43:05.334 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 242 k 2023-10-06T13:43:05.334 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 4.4 M 2023-10-06T13:43:05.334 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 18 M 2023-10-06T13:43:05.334 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-06T13:43:05.334 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-06T13:43:05.334 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-06T13:43:05.334 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-06T13:43:05.334 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-06T13:43:05.334 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-06T13:43:05.335 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-06T13:43:05.335 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-06T13:43:05.335 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-06T13:43:05.335 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-06T13:43:05.335 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-06T13:43:05.335 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-06T13:43:05.335 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-06T13:43:05.335 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-06T13:43:05.335 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-06T13:43:05.335 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-06T13:43:05.335 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-06T13:43:05.336 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-06T13:43:05.336 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-06T13:43:05.336 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-06T13:43:05.336 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-06T13:43:05.336 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-06T13:43:05.336 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-06T13:43:05.336 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-06T13:43:05.336 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-06T13:43:05.336 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-06T13:43:05.336 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-06T13:43:05.336 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:43:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================== 2023-10-06T13:43:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:Install 33 Packages 2023-10-06T13:43:05.337 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 30 M 2023-10-06T13:43:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 109 M 2023-10-06T13:43:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:43:05.706 INFO:teuthology.orchestra.run.smithi083.stdout:(1/33): ceph-18.2.0-596.g4af54c4a.el9.x86_64.rp 37 kB/s | 6.5 kB 00:00 2023-10-06T13:43:05.940 INFO:teuthology.orchestra.run.smithi083.stdout:(2/33): ceph-mds-18.2.0-596.g4af54c4a.el9.x86_6 5.1 MB/s | 2.1 MB 00:00 2023-10-06T13:43:05.991 INFO:teuthology.orchestra.run.smithi083.stdout:(3/33): ceph-mgr-18.2.0-596.g4af54c4a.el9.x86_6 3.1 MB/s | 1.4 MB 00:00 2023-10-06T13:43:06.098 INFO:teuthology.orchestra.run.smithi083.stdout:(4/33): ceph-mgr-modules-core-18.2.0-596.g4af54 2.2 MB/s | 242 kB 00:00 2023-10-06T13:43:06.199 INFO:teuthology.orchestra.run.smithi083.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 144 kB 00:00 2023-10-06T13:43:06.232 INFO:teuthology.orchestra.run.smithi083.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 944 kB/s | 32 kB 00:00 2023-10-06T13:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-test-2:18.2.0-596.g4af54c4a.el9.x86_64 6/6 2023-10-06T13:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:18.2.0-596.g4af54c4a.el9.x86_64 1/6 2023-10-06T13:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:18.2.0-596.g4af54c4a.el9.x86_64 2/6 2023-10-06T13:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-10-06T13:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2023-10-06T13:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2023-10-06T13:43:06.266 INFO:teuthology.orchestra.run.smithi083.stdout:(7/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2023-10-06T13:43:06.308 INFO:teuthology.orchestra.run.smithi083.stdout:(8/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2023-10-06T13:43:06.342 INFO:teuthology.orchestra.run.smithi083.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2023-10-06T13:43:06.377 INFO:teuthology.orchestra.run.smithi083.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 3.7 MB/s | 124 kB 00:00 2023-10-06T13:43:06.409 INFO:teuthology.orchestra.run.smithi083.stdout:(11/33): python3-requests-2.25.1-7.el9.noarch.r 3.3 MB/s | 113 kB 00:00 2023-10-06T13:43:06.451 INFO:teuthology.orchestra.run.smithi083.stdout:(12/33): python3-urllib3-1.26.5-3.el9.noarch.rp 4.4 MB/s | 188 kB 00:00 2023-10-06T13:43:06.626 INFO:teuthology.orchestra.run.smithi083.stdout:(13/33): ceph-mon-18.2.0-596.g4af54c4a.el9.x86_ 4.8 MB/s | 4.4 MB 00:00 2023-10-06T13:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : jq-1.6-15.el9.x86_64 6/6 2023-10-06T13:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: jq-1.6-15.el9.x86_64 2023-10-06T13:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-06T13:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-06T13:43:06.636 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-06T13:43:06.636 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:06.636 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:43:06.860 INFO:teuthology.orchestra.run.smithi083.stdout:(14/33): ceph-osd-18.2.0-596.g4af54c4a.el9.x86_ 19 MB/s | 18 MB 00:00 2023-10-06T13:43:06.878 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph 2023-10-06T13:43:07.119 INFO:teuthology.orchestra.run.smithi083.stdout:(15/33): python3-cryptography-36.0.1-4.el9.x86_ 1.8 MB/s | 1.2 MB 00:00 2023-10-06T13:43:07.169 INFO:teuthology.orchestra.run.smithi083.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 153 kB/s | 43 kB 00:00 2023-10-06T13:43:07.236 INFO:teuthology.orchestra.run.smithi083.stdout:(17/33): python3-pytz-2021.1-5.el9.noarch.rpm 84 kB/s | 51 kB 00:00 2023-10-06T13:43:07.428 INFO:teuthology.orchestra.run.smithi083.stdout:(18/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2023-10-06T13:43:07.453 INFO:teuthology.orchestra.run.smithi083.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 49 kB/s | 11 kB 00:00 2023-10-06T13:43:07.495 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:39 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:43:07.503 INFO:teuthology.orchestra.run.smithi083.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 235 kB/s | 18 kB 00:00 2023-10-06T13:43:07.529 INFO:teuthology.orchestra.run.smithi083.stdout:(21/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 419 kB/s | 172 kB 00:00 2023-10-06T13:43:07.554 INFO:teuthology.orchestra.run.smithi083.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 230 kB/s | 23 kB 00:00 2023-10-06T13:43:07.579 INFO:teuthology.orchestra.run.smithi083.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 257 kB/s | 19 kB 00:00 2023-10-06T13:43:07.604 INFO:teuthology.orchestra.run.smithi083.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2023-10-06T13:43:07.646 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:43:07.649 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================== 2023-10-06T13:43:07.649 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-10-06T13:43:07.649 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================== 2023-10-06T13:43:07.649 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:43:07.649 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 6.5 k 2023-10-06T13:43:07.649 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-10-06T13:43:07.650 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 2.1 M 2023-10-06T13:43:07.650 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 1.4 M 2023-10-06T13:43:07.650 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 242 k 2023-10-06T13:43:07.650 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 4.4 M 2023-10-06T13:43:07.650 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 18 M 2023-10-06T13:43:07.650 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-06T13:43:07.650 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-06T13:43:07.650 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-06T13:43:07.650 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-06T13:43:07.650 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-06T13:43:07.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-06T13:43:07.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-06T13:43:07.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-06T13:43:07.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-06T13:43:07.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-06T13:43:07.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-06T13:43:07.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-06T13:43:07.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-06T13:43:07.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-06T13:43:07.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-06T13:43:07.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-06T13:43:07.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-06T13:43:07.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-06T13:43:07.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-06T13:43:07.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-06T13:43:07.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-06T13:43:07.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-06T13:43:07.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-06T13:43:07.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-06T13:43:07.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-06T13:43:07.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-06T13:43:07.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-06T13:43:07.653 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:07.653 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:43:07.654 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================== 2023-10-06T13:43:07.654 INFO:teuthology.orchestra.run.smithi019.stdout:Install 33 Packages 2023-10-06T13:43:07.654 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:07.654 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 30 M 2023-10-06T13:43:07.654 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 109 M 2023-10-06T13:43:07.655 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:43:07.663 INFO:teuthology.orchestra.run.smithi083.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 945 kB/s | 79 kB 00:00 2023-10-06T13:43:07.688 INFO:teuthology.orchestra.run.smithi083.stdout:(26/33): python3-logutils-0.3.5-21.el9.noarch.r 345 kB/s | 46 kB 00:00 2023-10-06T13:43:07.739 INFO:teuthology.orchestra.run.smithi083.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 219 kB/s | 16 kB 00:00 2023-10-06T13:43:07.764 INFO:teuthology.orchestra.run.smithi083.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2023-10-06T13:43:07.814 INFO:teuthology.orchestra.run.smithi083.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 475 kB/s | 36 kB 00:00 2023-10-06T13:43:07.839 INFO:teuthology.orchestra.run.smithi083.stdout:(30/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 597 kB/s | 90 kB 00:00 2023-10-06T13:43:07.864 INFO:teuthology.orchestra.run.smithi083.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2023-10-06T13:43:07.915 INFO:teuthology.orchestra.run.smithi083.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 265 kB/s | 20 kB 00:00 2023-10-06T13:43:07.948 INFO:teuthology.orchestra.run.smithi083.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.1 MB/s | 427 kB 00:00 2023-10-06T13:43:07.954 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:Total 11 MB/s | 30 MB 00:02 2023-10-06T13:43:07.985 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:43:08.022 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:43:08.023 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:43:08.396 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:43:08.396 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:43:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:(1/33): ceph-18.2.0-596.g4af54c4a.el9.x86_64.rp 28 kB/s | 6.5 kB 00:00 2023-10-06T13:43:08.548 INFO:teuthology.orchestra.run.smithi019.stdout:(2/33): ceph-mds-18.2.0-596.g4af54c4a.el9.x86_6 5.5 MB/s | 2.1 MB 00:00 2023-10-06T13:43:08.582 INFO:teuthology.orchestra.run.smithi019.stdout:(3/33): ceph-mgr-18.2.0-596.g4af54c4a.el9.x86_6 3.5 MB/s | 1.4 MB 00:00 2023-10-06T13:43:08.673 INFO:teuthology.orchestra.run.smithi019.stdout:(4/33): ceph-mgr-modules-core-18.2.0-596.g4af54 2.6 MB/s | 242 kB 00:00 2023-10-06T13:43:08.782 INFO:teuthology.orchestra.run.smithi019.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2023-10-06T13:43:08.807 INFO:teuthology.orchestra.run.smithi019.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 1.2 MB/s | 32 kB 00:00 2023-10-06T13:43:08.833 INFO:teuthology.orchestra.run.smithi019.stdout:(7/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.5 MB/s | 39 kB 00:00 2023-10-06T13:43:09.985 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:43:09.985 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2023-10-06T13:43:09.985 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2023-10-06T13:43:09.986 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2023-10-06T13:43:09.986 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2023-10-06T13:43:09.986 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2023-10-06T13:43:09.986 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2023-10-06T13:43:09.987 INFO:teuthology.orchestra.run.smithi019.stdout:(8/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.7 MB/s | 241 kB 00:00 2023-10-06T13:43:09.987 INFO:teuthology.orchestra.run.smithi019.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2023-10-06T13:43:09.988 INFO:teuthology.orchestra.run.smithi019.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 3.6 MB/s | 124 kB 00:00 2023-10-06T13:43:09.988 INFO:teuthology.orchestra.run.smithi019.stdout:(11/33): python3-requests-2.25.1-7.el9.noarch.r 3.3 MB/s | 113 kB 00:00 2023-10-06T13:43:09.988 INFO:teuthology.orchestra.run.smithi019.stdout:(12/33): python3-urllib3-1.26.5-3.el9.noarch.rp 4.4 MB/s | 188 kB 00:00 2023-10-06T13:43:09.988 INFO:teuthology.orchestra.run.smithi019.stdout:(13/33): ceph-osd-18.2.0-596.g4af54c4a.el9.x86_ 21 MB/s | 18 MB 00:00 2023-10-06T13:43:09.988 INFO:teuthology.orchestra.run.smithi019.stdout:(14/33): python3-cryptography-36.0.1-4.el9.x86_ 3.1 MB/s | 1.2 MB 00:00 2023-10-06T13:43:09.988 INFO:teuthology.orchestra.run.smithi019.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 248 kB/s | 43 kB 00:00 2023-10-06T13:43:09.989 INFO:teuthology.orchestra.run.smithi019.stdout:(16/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2023-10-06T13:43:09.989 INFO:teuthology.orchestra.run.smithi019.stdout:(17/33): ceph-mon-18.2.0-596.g4af54c4a.el9.x86_ 3.1 MB/s | 4.4 MB 00:01 2023-10-06T13:43:09.989 INFO:teuthology.orchestra.run.smithi019.stdout:(18/33): python3-cherrypy-18.6.1-2.el9.noarch.r 2.8 MB/s | 358 kB 00:00 2023-10-06T13:43:09.989 INFO:teuthology.orchestra.run.smithi019.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 266 kB/s | 18 kB 00:00 2023-10-06T13:43:09.989 INFO:teuthology.orchestra.run.smithi019.stdout:(20/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 91 kB/s | 11 kB 00:00 2023-10-06T13:43:09.990 INFO:teuthology.orchestra.run.smithi019.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 395 kB/s | 23 kB 00:00 2023-10-06T13:43:09.999 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2023-10-06T13:43:10.012 INFO:teuthology.orchestra.run.smithi019.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2023-10-06T13:43:11.334 INFO:teuthology.orchestra.run.smithi019.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 293 kB/s | 20 kB 00:00 2023-10-06T13:43:11.334 INFO:teuthology.orchestra.run.smithi019.stdout:(24/33): python3-more-itertools-8.12.0-2.el9.no 1.2 MB/s | 79 kB 00:00 2023-10-06T13:43:11.334 INFO:teuthology.orchestra.run.smithi019.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 395 kB/s | 46 kB 00:00 2023-10-06T13:43:11.334 INFO:teuthology.orchestra.run.smithi019.stdout:(26/33): python3-pytz-2021.1-5.el9.noarch.rpm 66 kB/s | 51 kB 00:00 2023-10-06T13:43:11.334 INFO:teuthology.orchestra.run.smithi019.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2023-10-06T13:43:11.334 INFO:teuthology.orchestra.run.smithi019.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 217 kB/s | 16 kB 00:00 2023-10-06T13:43:11.334 INFO:teuthology.orchestra.run.smithi019.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 534 kB/s | 36 kB 00:00 2023-10-06T13:43:11.334 INFO:teuthology.orchestra.run.smithi019.stdout:(30/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.0 MB/s | 427 kB 00:00 2023-10-06T13:43:11.334 INFO:teuthology.orchestra.run.smithi019.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2023-10-06T13:43:11.335 INFO:teuthology.orchestra.run.smithi019.stdout:(32/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 398 kB/s | 90 kB 00:00 2023-10-06T13:43:11.335 INFO:teuthology.orchestra.run.smithi019.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 265 kB/s | 20 kB 00:00 2023-10-06T13:43:11.335 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:11.335 INFO:teuthology.orchestra.run.smithi019.stdout:Total 11 MB/s | 30 MB 00:02 2023-10-06T13:43:11.335 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:43:11.335 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:43:11.335 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:43:11.335 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:43:11.335 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:43:11.336 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 8/33 2023-10-06T13:43:11.336 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 9/33 2023-10-06T13:43:11.336 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 10/33 2023-10-06T13:43:11.336 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 11/33 2023-10-06T13:43:11.336 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 12/33 2023-10-06T13:43:11.336 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 13/33 2023-10-06T13:43:11.336 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 14/33 2023-10-06T13:43:11.336 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 15/33 2023-10-06T13:43:11.336 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 16/33 2023-10-06T13:43:11.336 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 17/33 2023-10-06T13:43:11.337 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 18/33 2023-10-06T13:43:11.337 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 19/33 2023-10-06T13:43:11.337 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 20/33 2023-10-06T13:43:11.337 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 21/33 2023-10-06T13:43:11.337 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2023-10-06T13:43:11.337 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2023-10-06T13:43:11.337 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2023-10-06T13:43:11.337 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2023-10-06T13:43:11.337 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2023-10-06T13:43:11.337 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-10-06T13:43:11.483 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-596.g4af54c4a.el9.n 28/33 2023-10-06T13:43:11.509 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:18.2.0-596.g4af54c4a.el9.x86_64 29/33 2023-10-06T13:43:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:18.2.0-596.g4af54c4a.el9.x86_64 29/33 2023-10-06T13:43:11.533 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:43:11.533 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-06T13:43:11.533 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-06T13:43:11.534 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-06T13:43:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:43:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2023-10-06T13:43:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2023-10-06T13:43:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2023-10-06T13:43:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2023-10-06T13:43:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2023-10-06T13:43:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2023-10-06T13:43:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2023-10-06T13:43:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 8/33 2023-10-06T13:43:12.872 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 9/33 2023-10-06T13:43:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 10/33 2023-10-06T13:43:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 11/33 2023-10-06T13:43:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 12/33 2023-10-06T13:43:13.069 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:18.2.0-596.g4af54c4a.el9.x86_64 30/33 2023-10-06T13:43:13.098 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:18.2.0-596.g4af54c4a.el9.x86_64 30/33 2023-10-06T13:43:13.098 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:43:13.098 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-06T13:43:13.099 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-06T13:43:13.099 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-06T13:43:13.099 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 13/33 2023-10-06T13:43:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 14/33 2023-10-06T13:43:13.335 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 15/33 2023-10-06T13:43:13.369 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 16/33 2023-10-06T13:43:13.399 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 17/33 2023-10-06T13:43:13.480 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 18/33 2023-10-06T13:43:13.510 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 19/33 2023-10-06T13:43:13.534 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 20/33 2023-10-06T13:43:13.553 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:18.2.0-596.g4af54c4a.el9.x86_64 31/33 2023-10-06T13:43:13.576 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:18.2.0-596.g4af54c4a.el9.x86_64 31/33 2023-10-06T13:43:13.576 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:43:13.576 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-06T13:43:13.576 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-06T13:43:13.577 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-06T13:43:13.577 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:13.577 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 21/33 2023-10-06T13:43:13.610 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2023-10-06T13:43:13.705 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2023-10-06T13:43:13.752 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2023-10-06T13:43:13.779 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:18.2.0-596.g4af54c4a.el9.x86_64 32/33 2023-10-06T13:43:13.801 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:18.2.0-596.g4af54c4a.el9.x86_64 32/33 2023-10-06T13:43:13.802 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:43:13.802 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-06T13:43:13.802 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-06T13:43:13.802 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-06T13:43:13.802 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:13.843 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2023-10-06T13:43:13.867 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:18.2.0-596.g4af54c4a.el9.x86_64 33/33 2023-10-06T13:43:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2023-10-06T13:43:14.135 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-10-06T13:43:14.304 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-596.g4af54c4a.el9.n 28/33 2023-10-06T13:43:14.330 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:18.2.0-596.g4af54c4a.el9.x86_64 29/33 2023-10-06T13:43:14.355 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:18.2.0-596.g4af54c4a.el9.x86_64 29/33 2023-10-06T13:43:14.355 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:43:14.355 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-06T13:43:14.355 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-06T13:43:14.356 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-06T13:43:14.356 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:15.742 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-2:18.2.0-596.g4af54c4a.el9.x86_64 33/33 2023-10-06T13:43:15.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:18.2.0-596.g4af54c4a.el9.x86_64 1/33 2023-10-06T13:43:15.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:18.2.0-596.g4af54c4a.el9.x86_64 2/33 2023-10-06T13:43:15.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:18.2.0-596.g4af54c4a.el9.x86_64 3/33 2023-10-06T13:43:15.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:18.2.0-596.g4af54c4a.el9.x86_64 4/33 2023-10-06T13:43:15.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:18.2.0-596.g4af54c4a.el9.x86_64 5/33 2023-10-06T13:43:15.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-596.g4af54c4a.el9.n 6/33 2023-10-06T13:43:15.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2023-10-06T13:43:15.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2023-10-06T13:43:15.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 9/33 2023-10-06T13:43:15.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 10/33 2023-10-06T13:43:15.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 11/33 2023-10-06T13:43:15.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 12/33 2023-10-06T13:43:15.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 13/33 2023-10-06T13:43:15.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 14/33 2023-10-06T13:43:15.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 15/33 2023-10-06T13:43:15.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/33 2023-10-06T13:43:15.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2023-10-06T13:43:15.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2023-10-06T13:43:15.746 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2023-10-06T13:43:15.746 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2023-10-06T13:43:15.746 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2023-10-06T13:43:15.746 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2023-10-06T13:43:15.746 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2023-10-06T13:43:15.746 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2023-10-06T13:43:15.746 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2023-10-06T13:43:15.747 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2023-10-06T13:43:15.747 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-10-06T13:43:15.747 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2023-10-06T13:43:15.747 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2023-10-06T13:43:15.747 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2023-10-06T13:43:15.747 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2023-10-06T13:43:15.747 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2023-10-06T13:43:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:18.2.0-596.g4af54c4a.el9.x86_64 30/33 2023-10-06T13:43:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:18.2.0-596.g4af54c4a.el9.x86_64 30/33 2023-10-06T13:43:16.003 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:43:16.003 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-06T13:43:16.003 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-06T13:43:16.003 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-06T13:43:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:16.473 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:18.2.0-596.g4af54c4a.el9.x86_64 31/33 2023-10-06T13:43:16.497 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:18.2.0-596.g4af54c4a.el9.x86_64 31/33 2023-10-06T13:43:16.498 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:43:16.498 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-06T13:43:16.498 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-06T13:43:16.498 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-06T13:43:16.498 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:16.707 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:18.2.0-596.g4af54c4a.el9.x86_64 32/33 2023-10-06T13:43:16.731 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:18.2.0-596.g4af54c4a.el9.x86_64 32/33 2023-10-06T13:43:16.732 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:43:16.732 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-06T13:43:16.732 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-06T13:43:16.732 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-06T13:43:16.732 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:16.799 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:18.2.0-596.g4af54c4a.el9.x86_64 33/33 2023-10-06T13:43:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2023-10-06T13:43:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:16.802 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:43:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-06T13:43:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-06T13:43:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-06T13:43:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-06T13:43:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-06T13:43:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-06T13:43:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-06T13:43:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-06T13:43:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-06T13:43:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-06T13:43:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-06T13:43:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-06T13:43:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-06T13:43:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-06T13:43:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-06T13:43:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2023-10-06T13:43:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-06T13:43:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-06T13:43:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-06T13:43:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-06T13:43:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-06T13:43:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-06T13:43:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-06T13:43:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-06T13:43:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-06T13:43:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-06T13:43:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-06T13:43:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:16.807 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:17.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-base 2023-10-06T13:43:17.729 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:50 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:17.817 INFO:teuthology.orchestra.run.smithi083.stdout:Package ceph-base-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:43:17.867 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:17.868 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2023-10-06T13:43:17.868 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:17.932 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install cephadm 2023-10-06T13:43:18.522 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:51 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:18.577 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-2:18.2.0-596.g4af54c4a.el9.x86_64 33/33 2023-10-06T13:43:18.577 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:18.2.0-596.g4af54c4a.el9.x86_64 1/33 2023-10-06T13:43:18.577 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:18.2.0-596.g4af54c4a.el9.x86_64 2/33 2023-10-06T13:43:18.577 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:18.2.0-596.g4af54c4a.el9.x86_64 3/33 2023-10-06T13:43:18.577 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:18.2.0-596.g4af54c4a.el9.x86_64 4/33 2023-10-06T13:43:18.577 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:18.2.0-596.g4af54c4a.el9.x86_64 5/33 2023-10-06T13:43:18.577 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-596.g4af54c4a.el9.n 6/33 2023-10-06T13:43:18.578 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2023-10-06T13:43:18.578 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2023-10-06T13:43:18.578 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 9/33 2023-10-06T13:43:18.578 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 10/33 2023-10-06T13:43:18.578 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 11/33 2023-10-06T13:43:18.578 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 12/33 2023-10-06T13:43:18.578 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 13/33 2023-10-06T13:43:18.578 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 14/33 2023-10-06T13:43:18.578 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 15/33 2023-10-06T13:43:18.578 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/33 2023-10-06T13:43:18.578 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2023-10-06T13:43:18.578 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2023-10-06T13:43:18.579 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2023-10-06T13:43:18.579 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2023-10-06T13:43:18.579 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2023-10-06T13:43:18.579 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2023-10-06T13:43:18.579 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2023-10-06T13:43:18.579 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2023-10-06T13:43:18.579 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2023-10-06T13:43:18.579 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2023-10-06T13:43:18.579 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-10-06T13:43:18.579 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2023-10-06T13:43:18.579 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2023-10-06T13:43:18.579 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2023-10-06T13:43:18.580 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2023-10-06T13:43:18.580 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2023-10-06T13:43:18.657 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:18.658 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:18.658 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2023-10-06T13:43:18.658 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:18.658 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:43:18.658 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 218 k 2023-10-06T13:43:18.658 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:18.658 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:43:18.658 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:18.658 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:43:18.659 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:18.659 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 218 k 2023-10-06T13:43:18.659 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 210 k 2023-10-06T13:43:18.659 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:43:19.170 INFO:teuthology.orchestra.run.smithi083.stdout:cephadm-18.2.0-596.g4af54c4a.el9.noarch.rpm 427 kB/s | 218 kB 00:00 2023-10-06T13:43:19.171 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:19.171 INFO:teuthology.orchestra.run.smithi083.stdout:Total 426 kB/s | 218 kB 00:00 2023-10-06T13:43:19.171 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:43:19.176 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:43:19.176 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:43:19.178 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:43:19.179 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:43:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:43:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2023-10-06T13:43:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:19.464 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:43:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-06T13:43:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-06T13:43:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-06T13:43:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-06T13:43:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-06T13:43:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-06T13:43:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-06T13:43:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-06T13:43:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-06T13:43:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-06T13:43:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-06T13:43:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-06T13:43:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-06T13:43:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-06T13:43:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-06T13:43:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2023-10-06T13:43:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-06T13:43:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-06T13:43:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-06T13:43:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-06T13:43:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-06T13:43:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-06T13:43:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-06T13:43:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-06T13:43:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-06T13:43:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-06T13:43:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-06T13:43:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:19.467 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:43:19.680 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 1/1 2023-10-06T13:43:19.757 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 1/1 2023-10-06T13:43:19.773 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-base 2023-10-06T13:43:20.094 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 1/1 2023-10-06T13:43:20.394 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:52 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:43:20.406 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 1/1 2023-10-06T13:43:20.406 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:20.406 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:43:20.406 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:20.406 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:20.406 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:20.485 INFO:teuthology.orchestra.run.smithi019.stdout:Package ceph-base-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:43:20.539 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:43:20.540 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-10-06T13:43:20.540 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:43:20.601 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install cephadm 2023-10-06T13:43:20.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-immutable-object-cache 2023-10-06T13:43:21.217 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:54 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:21.217 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:53 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:43:21.354 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:21.355 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:43:21.355 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:21.355 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:43:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 142 k 2023-10-06T13:43:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:21.355 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:43:21.355 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:21.355 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:43:21.355 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:21.356 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 142 k 2023-10-06T13:43:21.356 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 400 k 2023-10-06T13:43:21.356 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:43:21.358 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:43:21.359 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:21.359 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-10-06T13:43:21.359 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:21.359 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:43:21.359 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 218 k 2023-10-06T13:43:21.359 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:21.359 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:43:21.359 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:21.359 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:43:21.359 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:21.360 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 218 k 2023-10-06T13:43:21.360 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 210 k 2023-10-06T13:43:21.360 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:43:21.598 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm-18.2.0-596.g4af54c4a.el9.noarch.rpm 921 kB/s | 218 kB 00:00 2023-10-06T13:43:21.598 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:21.598 INFO:teuthology.orchestra.run.smithi019.stdout:Total 915 kB/s | 218 kB 00:00 2023-10-06T13:43:21.599 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:43:21.604 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:43:21.604 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:43:21.606 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:43:21.607 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:43:21.683 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:43:21.801 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-immutable-object-cache-18.2.0-596.g4af54c4 319 kB/s | 142 kB 00:00 2023-10-06T13:43:21.802 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:21.802 INFO:teuthology.orchestra.run.smithi083.stdout:Total 318 kB/s | 142 kB 00:00 2023-10-06T13:43:21.803 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:43:21.810 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:43:21.810 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:43:21.862 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:43:21.862 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:43:21.998 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:43:22.069 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-596.g4af54c4a.e 1/1 2023-10-06T13:43:22.085 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 1/1 2023-10-06T13:43:22.093 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-596.g4af54c4a.e 1/1 2023-10-06T13:43:22.093 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:43:22.093 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-06T13:43:22.094 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:22.171 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 1/1 2023-10-06T13:43:22.523 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 1/1 2023-10-06T13:43:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 1/1 2023-10-06T13:43:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:22.815 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:43:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:22.816 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:43:23.018 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-immutable-object-cache 2023-10-06T13:43:23.050 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-596.g4af54c4a.e 1/1 2023-10-06T13:43:23.050 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:23.050 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:43:23.050 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:23.050 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:23.050 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:23.293 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr 2023-10-06T13:43:23.638 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:55 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:43:23.783 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:43:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:43:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:43:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 142 k 2023-10-06T13:43:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:43:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:43:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:23.785 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 142 k 2023-10-06T13:43:23.785 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 400 k 2023-10-06T13:43:23.785 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:43:23.892 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:56 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:24.312 INFO:teuthology.orchestra.run.smithi083.stdout:Package ceph-mgr-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:43:24.312 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:24.312 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2023-10-06T13:43:24.312 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:24.312 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr-dashboard 2023-10-06T13:43:24.313 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-immutable-object-cache-18.2.0-596.g4af54c4 629 kB/s | 142 kB 00:00 2023-10-06T13:43:24.313 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:24.313 INFO:teuthology.orchestra.run.smithi019.stdout:Total 625 kB/s | 142 kB 00:00 2023-10-06T13:43:24.313 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:43:24.313 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:43:24.313 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:43:24.313 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:43:24.313 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:43:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:43:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-596.g4af54c4a.e 1/1 2023-10-06T13:43:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-596.g4af54c4a.e 1/1 2023-10-06T13:43:24.314 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:43:24.314 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-06T13:43:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:24.905 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:57 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:25.043 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2023-10-06T13:43:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:43:25.043 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 1.8 M 2023-10-06T13:43:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2023-10-06T13:43:25.043 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 23 k 2023-10-06T13:43:25.043 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 15 k 2023-10-06T13:43:25.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-06T13:43:25.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-06T13:43:25.043 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-06T13:43:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2023-10-06T13:43:25.044 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-06T13:43:25.044 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:43:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:Install 7 Packages 2023-10-06T13:43:25.044 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 2.1 M 2023-10-06T13:43:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 8.6 M 2023-10-06T13:43:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:43:25.281 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-596.g4af54c4a.e 1/1 2023-10-06T13:43:25.281 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:25.281 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:43:25.281 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:25.281 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:25.281 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:43:25.314 INFO:teuthology.orchestra.run.smithi083.stdout:(1/7): ceph-grafana-dashboards-18.2.0-596.g4af5 123 kB/s | 23 kB 00:00 2023-10-06T13:43:25.414 INFO:teuthology.orchestra.run.smithi083.stdout:(2/7): ceph-prometheus-alerts-18.2.0-596.g4af54 51 kB/s | 15 kB 00:00 2023-10-06T13:43:25.498 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr 2023-10-06T13:43:25.539 INFO:teuthology.orchestra.run.smithi083.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 40 kB/s | 9.0 kB 00:00 2023-10-06T13:43:25.698 INFO:teuthology.orchestra.run.smithi083.stdout:(4/7): ceph-mgr-dashboard-18.2.0-596.g4af54c4a. 3.1 MB/s | 1.8 MB 00:00 2023-10-06T13:43:25.739 INFO:teuthology.orchestra.run.smithi083.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 154 kB/s | 31 kB 00:00 2023-10-06T13:43:25.765 INFO:teuthology.orchestra.run.smithi083.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 117 kB/s | 41 kB 00:00 2023-10-06T13:43:26.120 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:58 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:43:26.123 INFO:teuthology.orchestra.run.smithi083.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 443 kB/s | 188 kB 00:00 2023-10-06T13:43:26.125 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:26.126 INFO:teuthology.orchestra.run.smithi083.stdout:Total 1.9 MB/s | 2.1 MB 00:01 2023-10-06T13:43:26.126 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:43:26.134 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:43:26.134 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:43:26.198 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:43:26.199 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:43:26.211 INFO:teuthology.orchestra.run.smithi019.stdout:Package ceph-mgr-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:43:26.262 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:43:26.263 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-10-06T13:43:26.263 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:43:26.326 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-dashboard 2023-10-06T13:43:26.494 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:43:26.542 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2023-10-06T13:43:26.575 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2023-10-06T13:43:26.621 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2023-10-06T13:43:26.654 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2023-10-06T13:43:26.682 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-596.g4af54c4a.el9.no 5/7 2023-10-06T13:43:26.916 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-596.g4af54c4a.el9.n 6/7 2023-10-06T13:43:26.944 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-596.g4af54c4a.el9.noarch 7/7 2023-10-06T13:43:26.950 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:58 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:43:27.094 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:43:27.094 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:27.094 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-10-06T13:43:27.095 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:27.095 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:43:27.095 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 1.8 M 2023-10-06T13:43:27.095 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-10-06T13:43:27.095 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 23 k 2023-10-06T13:43:27.095 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 15 k 2023-10-06T13:43:27.095 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-06T13:43:27.095 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-06T13:43:27.095 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-06T13:43:27.096 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2023-10-06T13:43:27.096 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-06T13:43:27.096 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:27.096 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:43:27.096 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:27.096 INFO:teuthology.orchestra.run.smithi019.stdout:Install 7 Packages 2023-10-06T13:43:27.096 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:27.097 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 2.1 M 2023-10-06T13:43:27.097 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 8.6 M 2023-10-06T13:43:27.097 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:43:27.498 INFO:teuthology.orchestra.run.smithi019.stdout:(1/7): ceph-grafana-dashboards-18.2.0-596.g4af5 119 kB/s | 23 kB 00:00 2023-10-06T13:43:27.557 INFO:teuthology.orchestra.run.smithi019.stdout:(2/7): ceph-prometheus-alerts-18.2.0-596.g4af54 58 kB/s | 15 kB 00:00 2023-10-06T13:43:27.577 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-596.g4af54c4a.el9.noarch 7/7 2023-10-06T13:43:27.577 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-596.g4af54c4a.el9.n 1/7 2023-10-06T13:43:27.577 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-596.g4af54c4a.el9.noarch 2/7 2023-10-06T13:43:27.577 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-596.g4af54c4a.el9.no 3/7 2023-10-06T13:43:27.577 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2023-10-06T13:43:27.577 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2023-10-06T13:43:27.577 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2023-10-06T13:43:27.607 INFO:teuthology.orchestra.run.smithi019.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 83 kB/s | 9.0 kB 00:00 2023-10-06T13:43:27.641 INFO:teuthology.orchestra.run.smithi019.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 495 kB/s | 41 kB 00:00 2023-10-06T13:43:27.675 INFO:teuthology.orchestra.run.smithi019.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 458 kB/s | 31 kB 00:00 2023-10-06T13:43:27.708 INFO:teuthology.orchestra.run.smithi019.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 2.7 MB/s | 188 kB 00:00 2023-10-06T13:43:27.899 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2023-10-06T13:43:27.899 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:27.899 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:43:27.899 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:27.899 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:27.899 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:27.899 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-06T13:43:27.899 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-06T13:43:27.899 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-06T13:43:27.899 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-06T13:43:27.900 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:27.900 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:28.075 INFO:teuthology.orchestra.run.smithi019.stdout:(7/7): ceph-mgr-dashboard-18.2.0-596.g4af54c4a. 2.3 MB/s | 1.8 MB 00:00 2023-10-06T13:43:28.079 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:28.079 INFO:teuthology.orchestra.run.smithi019.stdout:Total 2.1 MB/s | 2.1 MB 00:00 2023-10-06T13:43:28.079 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:43:28.087 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:43:28.088 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:43:28.156 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:43:28.156 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:43:28.162 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr-diskprediction-local 2023-10-06T13:43:28.467 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:43:28.543 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2023-10-06T13:43:28.577 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2023-10-06T13:43:28.625 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2023-10-06T13:43:28.664 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2023-10-06T13:43:28.693 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-596.g4af54c4a.el9.no 5/7 2023-10-06T13:43:28.745 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:01 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:28.884 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:28.885 INFO:teuthology.orchestra.run.smithi083.stdout:=========================================================================================== 2023-10-06T13:43:28.885 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2023-10-06T13:43:28.885 INFO:teuthology.orchestra.run.smithi083.stdout:=========================================================================================== 2023-10-06T13:43:28.885 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:43:28.885 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 7.4 M 2023-10-06T13:43:28.886 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2023-10-06T13:43:28.886 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-06T13:43:28.886 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-06T13:43:28.886 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-06T13:43:28.886 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-06T13:43:28.886 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-06T13:43:28.886 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-06T13:43:28.886 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-06T13:43:28.886 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-06T13:43:28.886 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-06T13:43:28.886 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-06T13:43:28.887 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-06T13:43:28.887 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:28.887 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:43:28.887 INFO:teuthology.orchestra.run.smithi083.stdout:=========================================================================================== 2023-10-06T13:43:28.887 INFO:teuthology.orchestra.run.smithi083.stdout:Install 12 Packages 2023-10-06T13:43:28.887 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:28.888 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 37 M 2023-10-06T13:43:28.888 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 209 M 2023-10-06T13:43:28.888 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:43:28.941 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-596.g4af54c4a.el9.n 6/7 2023-10-06T13:43:28.979 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-596.g4af54c4a.el9.noarch 7/7 2023-10-06T13:43:29.022 INFO:teuthology.orchestra.run.smithi083.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 235 kB/s | 32 kB 00:00 2023-10-06T13:43:29.048 INFO:teuthology.orchestra.run.smithi083.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 723 kB/s | 17 kB 00:00 2023-10-06T13:43:29.081 INFO:teuthology.orchestra.run.smithi083.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2023-10-06T13:43:29.131 INFO:teuthology.orchestra.run.smithi083.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2023-10-06T13:43:29.199 INFO:teuthology.orchestra.run.smithi083.stdout:(5/12): python3-devel-3.9.17-2.el9.x86_64.rpm 3.0 MB/s | 205 kB 00:00 2023-10-06T13:43:29.274 INFO:teuthology.orchestra.run.smithi083.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 25 MB/s | 4.7 MB 00:00 2023-10-06T13:43:29.366 INFO:teuthology.orchestra.run.smithi083.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2023-10-06T13:43:29.391 INFO:teuthology.orchestra.run.smithi083.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.9 MB/s | 217 kB 00:00 2023-10-06T13:43:29.513 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-596.g4af54c4a.el9.noarch 7/7 2023-10-06T13:43:29.513 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-596.g4af54c4a.el9.n 1/7 2023-10-06T13:43:29.513 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-596.g4af54c4a.el9.noarch 2/7 2023-10-06T13:43:29.513 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-596.g4af54c4a.el9.no 3/7 2023-10-06T13:43:29.513 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2023-10-06T13:43:29.513 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2023-10-06T13:43:29.513 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2023-10-06T13:43:29.717 INFO:teuthology.orchestra.run.smithi083.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2023-10-06T13:43:29.750 INFO:teuthology.orchestra.run.smithi083.stdout:(10/12): libgfortran-11.4.1-2.3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2023-10-06T13:43:29.775 INFO:teuthology.orchestra.run.smithi083.stdout:(11/12): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2023-10-06T13:43:29.832 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2023-10-06T13:43:29.833 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:29.833 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:43:29.833 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:29.833 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:29.833 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:29.833 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-06T13:43:29.833 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-06T13:43:29.833 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-06T13:43:29.833 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-06T13:43:29.833 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:29.833 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:43:30.065 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-diskprediction-local 2023-10-06T13:43:30.567 INFO:teuthology.orchestra.run.smithi083.stdout:(12/12): ceph-mgr-diskprediction-local-18.2.0-5 4.4 MB/s | 7.4 MB 00:01 2023-10-06T13:43:30.570 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:30.570 INFO:teuthology.orchestra.run.smithi083.stdout:Total 22 MB/s | 37 MB 00:01 2023-10-06T13:43:30.570 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:43:30.591 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:43:30.592 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:43:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:02 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:43:30.829 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:43:30.830 INFO:teuthology.orchestra.run.smithi019.stdout:=========================================================================================== 2023-10-06T13:43:30.830 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-10-06T13:43:30.830 INFO:teuthology.orchestra.run.smithi019.stdout:=========================================================================================== 2023-10-06T13:43:30.830 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:43:30.830 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 7.4 M 2023-10-06T13:43:30.831 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-10-06T13:43:30.831 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-06T13:43:30.831 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-06T13:43:30.831 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-06T13:43:30.831 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-06T13:43:30.831 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-06T13:43:30.831 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-06T13:43:30.831 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-06T13:43:30.831 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-06T13:43:30.831 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-06T13:43:30.831 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-06T13:43:30.832 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-06T13:43:30.832 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:30.832 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:43:30.832 INFO:teuthology.orchestra.run.smithi019.stdout:=========================================================================================== 2023-10-06T13:43:30.832 INFO:teuthology.orchestra.run.smithi019.stdout:Install 12 Packages 2023-10-06T13:43:30.832 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:30.832 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 37 M 2023-10-06T13:43:30.833 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 209 M 2023-10-06T13:43:30.833 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:43:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 240 kB/s | 32 kB 00:00 2023-10-06T13:43:30.991 INFO:teuthology.orchestra.run.smithi019.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 720 kB/s | 17 kB 00:00 2023-10-06T13:43:31.015 INFO:teuthology.orchestra.run.smithi019.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.4 MB/s | 34 kB 00:00 2023-10-06T13:43:31.027 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:43:31.027 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:43:31.099 INFO:teuthology.orchestra.run.smithi019.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2023-10-06T13:43:31.175 INFO:teuthology.orchestra.run.smithi019.stdout:(5/12): python3-devel-3.9.17-2.el9.x86_64.rpm 2.7 MB/s | 205 kB 00:00 2023-10-06T13:43:31.242 INFO:teuthology.orchestra.run.smithi019.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2023-10-06T13:43:31.325 INFO:teuthology.orchestra.run.smithi019.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2023-10-06T13:43:31.351 INFO:teuthology.orchestra.run.smithi019.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 217 kB 00:00 2023-10-06T13:43:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2023-10-06T13:43:31.701 INFO:teuthology.orchestra.run.smithi019.stdout:(10/12): libgfortran-11.4.1-2.3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2023-10-06T13:43:31.718 INFO:teuthology.orchestra.run.smithi019.stdout:(11/12): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2023-10-06T13:43:31.737 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:43:31.818 INFO:teuthology.orchestra.run.smithi019.stdout:(12/12): ceph-mgr-diskprediction-local-18.2.0-5 7.5 MB/s | 7.4 MB 00:00 2023-10-06T13:43:31.835 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 1/12 2023-10-06T13:43:31.836 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:31.836 INFO:teuthology.orchestra.run.smithi019.stdout:Total 37 MB/s | 37 MB 00:00 2023-10-06T13:43:31.836 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:43:31.844 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:43:31.932 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:43:31.932 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 2/12 2023-10-06T13:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2023-10-06T13:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2023-10-06T13:43:32.160 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2023-10-06T13:43:32.190 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-10-06T13:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2023-10-06T13:43:32.286 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:43:32.287 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:43:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2023-10-06T13:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2023-10-06T13:43:33.136 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:43:33.200 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 1/12 2023-10-06T13:43:33.231 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 2/12 2023-10-06T13:43:33.330 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2023-10-06T13:43:33.376 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2023-10-06T13:43:33.560 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2023-10-06T13:43:33.591 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-10-06T13:43:33.678 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2023-10-06T13:43:34.282 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2023-10-06T13:43:34.355 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2023-10-06T13:43:34.467 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2023-10-06T13:43:35.311 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2023-10-06T13:43:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-596.g4af54c 12/12 2023-10-06T13:43:35.794 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2023-10-06T13:43:36.879 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2023-10-06T13:43:36.907 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-596.g4af54c 12/12 2023-10-06T13:43:37.943 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-596.g4af54c 12/12 2023-10-06T13:43:37.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-596.g4af54c 1/12 2023-10-06T13:43:37.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2023-10-06T13:43:37.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2023-10-06T13:43:37.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2023-10-06T13:43:37.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2023-10-06T13:43:37.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-10-06T13:43:37.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 7/12 2023-10-06T13:43:37.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 8/12 2023-10-06T13:43:37.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 9/12 2023-10-06T13:43:37.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 10/12 2023-10-06T13:43:37.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 11/12 2023-10-06T13:43:38.409 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 12/12 2023-10-06T13:43:38.409 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:38.410 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:43:38.410 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:38.410 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-06T13:43:38.410 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-06T13:43:38.410 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-06T13:43:38.410 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-06T13:43:38.410 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-06T13:43:38.410 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-06T13:43:38.410 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-06T13:43:38.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-06T13:43:38.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-06T13:43:38.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-06T13:43:38.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-06T13:43:38.411 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:38.411 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:38.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr-rook 2023-10-06T13:43:39.408 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:12 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-596.g4af54c 12/12 2023-10-06T13:43:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-596.g4af54c 1/12 2023-10-06T13:43:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2023-10-06T13:43:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2023-10-06T13:43:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2023-10-06T13:43:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2023-10-06T13:43:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-10-06T13:43:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 7/12 2023-10-06T13:43:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 8/12 2023-10-06T13:43:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 9/12 2023-10-06T13:43:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 10/12 2023-10-06T13:43:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 11/12 2023-10-06T13:43:39.545 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:39.546 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2023-10-06T13:43:39.546 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2023-10-06T13:43:39.546 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2023-10-06T13:43:39.546 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:43:39.546 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 47 k 2023-10-06T13:43:39.546 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2023-10-06T13:43:39.547 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-06T13:43:39.547 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-06T13:43:39.547 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.23.0-1.el9 epel 196 k 2023-10-06T13:43:39.547 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-06T13:43:39.547 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-06T13:43:39.547 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-06T13:43:39.547 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-06T13:43:39.547 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-06T13:43:39.547 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-06T13:43:39.547 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-06T13:43:39.547 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-06T13:43:39.547 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-06T13:43:39.548 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:39.548 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:43:39.548 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2023-10-06T13:43:39.548 INFO:teuthology.orchestra.run.smithi083.stdout:Install 13 Packages 2023-10-06T13:43:39.548 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:39.548 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 2.0 M 2023-10-06T13:43:39.548 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 26 M 2023-10-06T13:43:39.549 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:43:39.766 INFO:teuthology.orchestra.run.smithi083.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 204 kB/s | 24 kB 00:00 2023-10-06T13:43:39.783 INFO:teuthology.orchestra.run.smithi083.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 133 kB/s | 18 kB 00:00 2023-10-06T13:43:39.833 INFO:teuthology.orchestra.run.smithi083.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 174 kB 00:00 2023-10-06T13:43:39.858 INFO:teuthology.orchestra.run.smithi083.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2023-10-06T13:43:39.884 INFO:teuthology.orchestra.run.smithi083.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2023-10-06T13:43:39.909 INFO:teuthology.orchestra.run.smithi083.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 854 kB/s | 43 kB 00:00 2023-10-06T13:43:39.943 INFO:teuthology.orchestra.run.smithi083.stdout:(7/13): ceph-mgr-rook-18.2.0-596.g4af54c4a.el9. 160 kB/s | 47 kB 00:00 2023-10-06T13:43:40.027 INFO:teuthology.orchestra.run.smithi083.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 128 kB/s | 15 kB 00:00 2023-10-06T13:43:40.052 INFO:teuthology.orchestra.run.smithi083.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 192 kB/s | 32 kB 00:00 2023-10-06T13:43:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 12/12 2023-10-06T13:43:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:40.065 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:43:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-06T13:43:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-06T13:43:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-06T13:43:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-06T13:43:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-06T13:43:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-06T13:43:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-06T13:43:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-06T13:43:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-06T13:43:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-06T13:43:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-06T13:43:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:40.067 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:43:40.152 INFO:teuthology.orchestra.run.smithi083.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 591 kB/s | 59 kB 00:00 2023-10-06T13:43:40.177 INFO:teuthology.orchestra.run.smithi083.stdout:(11/13): python3-google-auth-2.23.0-1.el9.noarc 836 kB/s | 196 kB 00:00 2023-10-06T13:43:40.236 INFO:teuthology.orchestra.run.smithi083.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2023-10-06T13:43:40.295 INFO:teuthology.orchestra.run.smithi083.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.8 MB/s | 1.0 MB 00:00 2023-10-06T13:43:40.296 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:40.297 INFO:teuthology.orchestra.run.smithi083.stdout:Total 2.7 MB/s | 2.0 MB 00:00 2023-10-06T13:43:40.297 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:43:40.310 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:43:40.310 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:43:40.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-rook 2023-10-06T13:43:40.404 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:43:40.404 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:43:40.938 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:43:41.019 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:13 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:43:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2023-10-06T13:43:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2023-10-06T13:43:41.164 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:43:41.166 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2023-10-06T13:43:41.166 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-10-06T13:43:41.166 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2023-10-06T13:43:41.166 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:43:41.166 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 47 k 2023-10-06T13:43:41.166 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-10-06T13:43:41.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-06T13:43:41.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-06T13:43:41.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.23.0-1.el9 epel 196 k 2023-10-06T13:43:41.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-06T13:43:41.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-06T13:43:41.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-06T13:43:41.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-06T13:43:41.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-06T13:43:41.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-06T13:43:41.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-06T13:43:41.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-06T13:43:41.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-06T13:43:41.168 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:41.168 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:43:41.168 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2023-10-06T13:43:41.168 INFO:teuthology.orchestra.run.smithi019.stdout:Install 13 Packages 2023-10-06T13:43:41.168 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:41.168 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 2.0 M 2023-10-06T13:43:41.169 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 26 M 2023-10-06T13:43:41.169 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:43:41.226 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2023-10-06T13:43:41.255 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2023-10-06T13:43:41.296 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2023-10-06T13:43:41.389 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2023-10-06T13:43:41.492 INFO:teuthology.orchestra.run.smithi019.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 153 kB/s | 18 kB 00:00 2023-10-06T13:43:41.498 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.23.0-1.el9.noarch 7/13 2023-10-06T13:43:41.517 INFO:teuthology.orchestra.run.smithi019.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 170 kB/s | 24 kB 00:00 2023-10-06T13:43:41.556 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2023-10-06T13:43:41.558 INFO:teuthology.orchestra.run.smithi019.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 174 kB 00:00 2023-10-06T13:43:41.593 INFO:teuthology.orchestra.run.smithi019.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2023-10-06T13:43:41.618 INFO:teuthology.orchestra.run.smithi019.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.5 MB/s | 211 kB 00:00 2023-10-06T13:43:41.643 INFO:teuthology.orchestra.run.smithi019.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 857 kB/s | 43 kB 00:00 2023-10-06T13:43:41.710 INFO:teuthology.orchestra.run.smithi019.stdout:(7/13): ceph-mgr-rook-18.2.0-596.g4af54c4a.el9. 141 kB/s | 47 kB 00:00 2023-10-06T13:43:41.744 INFO:teuthology.orchestra.run.smithi019.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 149 kB/s | 15 kB 00:00 2023-10-06T13:43:41.777 INFO:teuthology.orchestra.run.smithi019.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 202 kB/s | 32 kB 00:00 2023-10-06T13:43:42.069 INFO:teuthology.orchestra.run.smithi019.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 202 kB/s | 59 kB 00:00 2023-10-06T13:43:42.094 INFO:teuthology.orchestra.run.smithi019.stdout:(11/13): python3-google-auth-2.23.0-1.el9.noarc 510 kB/s | 196 kB 00:00 2023-10-06T13:43:42.153 INFO:teuthology.orchestra.run.smithi019.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 2.5 MB/s | 1.0 MB 00:00 2023-10-06T13:43:42.178 INFO:teuthology.orchestra.run.smithi019.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 825 kB/s | 90 kB 00:00 2023-10-06T13:43:42.180 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:42.180 INFO:teuthology.orchestra.run.smithi019.stdout:Total 2.0 MB/s | 2.0 MB 00:01 2023-10-06T13:43:42.181 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:43:42.196 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:43:42.196 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:43:42.273 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2023-10-06T13:43:42.304 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:43:42.305 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:43:42.321 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2023-10-06T13:43:42.354 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2023-10-06T13:43:42.400 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2023-10-06T13:43:42.424 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:18.2.0-596.g4af54c4a.el9.noarch 13/13 2023-10-06T13:43:42.828 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:43:43.000 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2023-10-06T13:43:43.056 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2023-10-06T13:43:43.112 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2023-10-06T13:43:43.148 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2023-10-06T13:43:43.190 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2023-10-06T13:43:43.218 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-596.g4af54c4a.el9.noarch 13/13 2023-10-06T13:43:43.218 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:18.2.0-596.g4af54c4a.el9.noarch 1/13 2023-10-06T13:43:43.218 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2023-10-06T13:43:43.218 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2023-10-06T13:43:43.218 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2023-10-06T13:43:43.219 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2023-10-06T13:43:43.219 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2023-10-06T13:43:43.219 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2023-10-06T13:43:43.219 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2023-10-06T13:43:43.219 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2023-10-06T13:43:43.219 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.23.0-1.el9.noarch 10/13 2023-10-06T13:43:43.219 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2023-10-06T13:43:43.219 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2023-10-06T13:43:43.285 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2023-10-06T13:43:43.396 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.23.0-1.el9.noarch 7/13 2023-10-06T13:43:43.450 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2023-10-06T13:43:43.616 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2023-10-06T13:43:43.616 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:43.616 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:43:43.616 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:43.616 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-06T13:43:43.617 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-06T13:43:43.617 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.23.0-1.el9.noarch 2023-10-06T13:43:43.617 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-06T13:43:43.617 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-06T13:43:43.617 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-06T13:43:43.617 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-06T13:43:43.617 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-06T13:43:43.617 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-06T13:43:43.617 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-06T13:43:43.617 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-06T13:43:43.617 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-06T13:43:43.617 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:43.618 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:43.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr-cephadm 2023-10-06T13:43:44.220 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2023-10-06T13:43:44.271 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2023-10-06T13:43:44.304 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2023-10-06T13:43:44.350 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2023-10-06T13:43:44.374 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:18.2.0-596.g4af54c4a.el9.noarch 13/13 2023-10-06T13:43:44.450 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:17 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:44.589 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:44.590 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2023-10-06T13:43:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2023-10-06T13:43:44.590 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2023-10-06T13:43:44.590 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:43:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 132 k 2023-10-06T13:43:44.590 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2023-10-06T13:43:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-06T13:43:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-06T13:43:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-06T13:43:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-06T13:43:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-06T13:43:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:44.591 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:43:44.591 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2023-10-06T13:43:44.591 INFO:teuthology.orchestra.run.smithi083.stdout:Install 6 Packages 2023-10-06T13:43:44.591 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:44.592 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 6.8 M 2023-10-06T13:43:44.592 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 33 M 2023-10-06T13:43:44.592 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:43:44.923 INFO:teuthology.orchestra.run.smithi083.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.4 MB/s | 228 kB 00:00 2023-10-06T13:43:45.099 INFO:teuthology.orchestra.run.smithi083.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2023-10-06T13:43:45.190 INFO:teuthology.orchestra.run.smithi083.stdout:(3/6): ceph-mgr-cephadm-18.2.0-596.g4af54c4a.el 306 kB/s | 132 kB 00:00 2023-10-06T13:43:45.205 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-596.g4af54c4a.el9.noarch 13/13 2023-10-06T13:43:45.205 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:18.2.0-596.g4af54c4a.el9.noarch 1/13 2023-10-06T13:43:45.205 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2023-10-06T13:43:45.206 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2023-10-06T13:43:45.206 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2023-10-06T13:43:45.206 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2023-10-06T13:43:45.206 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2023-10-06T13:43:45.206 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2023-10-06T13:43:45.206 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2023-10-06T13:43:45.206 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2023-10-06T13:43:45.206 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.23.0-1.el9.noarch 10/13 2023-10-06T13:43:45.207 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2023-10-06T13:43:45.207 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2023-10-06T13:43:45.316 INFO:teuthology.orchestra.run.smithi083.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 266 kB/s | 58 kB 00:00 2023-10-06T13:43:45.358 INFO:teuthology.orchestra.run.smithi083.stdout:(5/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 1.2 MB/s | 548 kB 00:00 2023-10-06T13:43:45.391 INFO:teuthology.orchestra.run.smithi083.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 254 kB/s | 51 kB 00:00 2023-10-06T13:43:45.393 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:45.393 INFO:teuthology.orchestra.run.smithi083.stdout:Total 8.4 MB/s | 6.8 MB 00:00 2023-10-06T13:43:45.393 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:43:45.402 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:43:45.402 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:43:45.491 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:43:45.491 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:43:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2023-10-06T13:43:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:45.595 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:43:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-06T13:43:45.596 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-06T13:43:45.596 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.23.0-1.el9.noarch 2023-10-06T13:43:45.596 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-06T13:43:45.596 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-06T13:43:45.596 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-06T13:43:45.596 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-06T13:43:45.596 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-06T13:43:45.596 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-06T13:43:45.596 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-06T13:43:45.596 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-06T13:43:45.596 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-06T13:43:45.597 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:45.597 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:43:45.790 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:43:45.836 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-cephadm 2023-10-06T13:43:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2023-10-06T13:43:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2023-10-06T13:43:46.456 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:18 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:43:46.511 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2023-10-06T13:43:46.597 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2023-10-06T13:43:46.602 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:43:46.603 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2023-10-06T13:43:46.604 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-10-06T13:43:46.604 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2023-10-06T13:43:46.604 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:43:46.604 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 132 k 2023-10-06T13:43:46.604 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-10-06T13:43:46.604 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-06T13:43:46.604 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-06T13:43:46.604 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-06T13:43:46.604 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-06T13:43:46.604 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-06T13:43:46.604 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:46.605 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:43:46.605 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2023-10-06T13:43:46.605 INFO:teuthology.orchestra.run.smithi019.stdout:Install 6 Packages 2023-10-06T13:43:46.605 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:46.605 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 6.8 M 2023-10-06T13:43:46.606 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 33 M 2023-10-06T13:43:46.606 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:43:46.666 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2023-10-06T13:43:46.691 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 6/6 2023-10-06T13:43:46.971 INFO:teuthology.orchestra.run.smithi019.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.3 MB/s | 228 kB 00:00 2023-10-06T13:43:47.130 INFO:teuthology.orchestra.run.smithi019.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2023-10-06T13:43:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:(3/6): ceph-mgr-cephadm-18.2.0-596.g4af54c4a.el 325 kB/s | 132 kB 00:00 2023-10-06T13:43:47.305 INFO:teuthology.orchestra.run.smithi019.stdout:(4/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 1.6 MB/s | 548 kB 00:00 2023-10-06T13:43:47.330 INFO:teuthology.orchestra.run.smithi019.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 288 kB/s | 58 kB 00:00 2023-10-06T13:43:47.381 INFO:teuthology.orchestra.run.smithi019.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 291 kB/s | 51 kB 00:00 2023-10-06T13:43:47.382 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:47.383 INFO:teuthology.orchestra.run.smithi019.stdout:Total 8.7 MB/s | 6.8 MB 00:00 2023-10-06T13:43:47.383 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:43:47.393 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:43:47.393 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:43:47.470 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 6/6 2023-10-06T13:43:47.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 1/6 2023-10-06T13:43:47.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2023-10-06T13:43:47.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2023-10-06T13:43:47.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2023-10-06T13:43:47.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2023-10-06T13:43:47.488 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:43:47.488 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:43:47.787 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2023-10-06T13:43:47.787 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:47.787 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:43:47.787 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:47.787 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-06T13:43:47.787 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-06T13:43:47.787 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-06T13:43:47.788 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-06T13:43:47.788 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-06T13:43:47.788 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:47.788 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:47.788 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:43:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2023-10-06T13:43:48.016 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2023-10-06T13:43:48.047 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-fuse 2023-10-06T13:43:48.564 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2023-10-06T13:43:48.634 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:21 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:48.662 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2023-10-06T13:43:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2023-10-06T13:43:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 6/6 2023-10-06T13:43:48.772 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:48.772 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:48.772 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2023-10-06T13:43:48.772 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:48.773 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:43:48.773 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 836 k 2023-10-06T13:43:48.773 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2023-10-06T13:43:48.773 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-06T13:43:48.773 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:48.773 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:43:48.773 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:48.773 INFO:teuthology.orchestra.run.smithi083.stdout:Install 2 Packages 2023-10-06T13:43:48.773 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:48.773 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 914 k 2023-10-06T13:43:48.773 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 2.7 M 2023-10-06T13:43:48.773 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:43:49.004 INFO:teuthology.orchestra.run.smithi083.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 341 kB/s | 79 kB 00:00 2023-10-06T13:43:49.062 INFO:teuthology.orchestra.run.smithi083.stdout:(2/2): ceph-fuse-18.2.0-596.g4af54c4a.el9.x86_6 2.8 MB/s | 836 kB 00:00 2023-10-06T13:43:49.064 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:49.064 INFO:teuthology.orchestra.run.smithi083.stdout:Total 3.1 MB/s | 914 kB 00:00 2023-10-06T13:43:49.064 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:43:49.085 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:43:49.085 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:43:49.158 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:43:49.158 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:43:49.336 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:43:49.467 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2023-10-06T13:43:49.545 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 2/2 2023-10-06T13:43:49.573 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 6/6 2023-10-06T13:43:49.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 1/6 2023-10-06T13:43:49.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2023-10-06T13:43:49.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2023-10-06T13:43:49.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2023-10-06T13:43:49.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2023-10-06T13:43:49.897 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2023-10-06T13:43:49.898 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:49.898 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:43:49.898 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:49.898 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-06T13:43:49.898 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-06T13:43:49.898 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-06T13:43:49.898 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-06T13:43:49.898 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-06T13:43:49.898 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:49.898 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:43:50.156 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-fuse 2023-10-06T13:43:50.210 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 2/2 2023-10-06T13:43:50.210 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 1/2 2023-10-06T13:43:50.527 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2023-10-06T13:43:50.527 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:50.527 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:43:50.527 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2023-10-06T13:43:50.527 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:50.527 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:50.832 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:22 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:43:50.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-volume 2023-10-06T13:43:50.975 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:43:50.976 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:50.976 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-10-06T13:43:50.976 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:50.976 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:43:50.976 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 836 k 2023-10-06T13:43:50.976 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-10-06T13:43:50.976 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-06T13:43:50.976 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:50.977 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:43:50.977 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:50.977 INFO:teuthology.orchestra.run.smithi019.stdout:Install 2 Packages 2023-10-06T13:43:50.977 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:50.977 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 914 k 2023-10-06T13:43:50.977 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 2.7 M 2023-10-06T13:43:50.978 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:43:51.090 INFO:teuthology.orchestra.run.smithi019.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 697 kB/s | 79 kB 00:00 2023-10-06T13:43:51.424 INFO:teuthology.orchestra.run.smithi019.stdout:(2/2): ceph-fuse-18.2.0-596.g4af54c4a.el9.x86_6 1.8 MB/s | 836 kB 00:00 2023-10-06T13:43:51.425 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:51.425 INFO:teuthology.orchestra.run.smithi019.stdout:Total 2.0 MB/s | 914 kB 00:00 2023-10-06T13:43:51.425 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:43:51.447 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:43:51.447 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:43:51.448 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:24 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:51.526 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:43:51.526 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:43:51.588 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:51.588 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2023-10-06T13:43:51.589 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:51.589 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:43:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 262 k 2023-10-06T13:43:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:51.589 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:43:51.589 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:51.589 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:43:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:51.589 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 262 k 2023-10-06T13:43:51.589 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 1.2 M 2023-10-06T13:43:51.589 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:43:51.719 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:43:51.858 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2023-10-06T13:43:51.934 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 2/2 2023-10-06T13:43:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-volume-18.2.0-596.g4af54c4a.el9.noarch.rpm 761 kB/s | 262 kB 00:00 2023-10-06T13:43:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:Total 758 kB/s | 262 kB 00:00 2023-10-06T13:43:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:43:51.943 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:43:51.943 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:43:51.973 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:43:51.974 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:43:52.186 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:43:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:18.2.0-596.g4af54c4a.el9.noarch 1/1 2023-10-06T13:43:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:18.2.0-596.g4af54c4a.el9.noarch 1/1 2023-10-06T13:43:52.277 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:43:52.277 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-06T13:43:52.278 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:52.667 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 2/2 2023-10-06T13:43:52.667 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 1/2 2023-10-06T13:43:52.983 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2023-10-06T13:43:52.983 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:52.983 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:43:52.983 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2023-10-06T13:43:52.983 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:52.983 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:43:53.289 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-volume 2023-10-06T13:43:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:18.2.0-596.g4af54c4a.el9.noarch 1/1 2023-10-06T13:43:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:53.332 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:43:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:53.333 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:53.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install librados-devel 2023-10-06T13:43:53.917 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:25 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:43:54.061 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:43:54.061 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:54.061 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-10-06T13:43:54.061 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:54.062 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:43:54.062 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:18.2.0-596.g4af54c4a.el9 ceph-noarch 262 k 2023-10-06T13:43:54.062 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:54.062 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:43:54.062 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:54.062 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:43:54.062 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:54.062 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 262 k 2023-10-06T13:43:54.062 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 1.2 M 2023-10-06T13:43:54.062 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:43:54.275 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:27 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:54.411 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-volume-18.2.0-596.g4af54c4a.el9.noarch.rpm 751 kB/s | 262 kB 00:00 2023-10-06T13:43:54.412 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:54.412 INFO:teuthology.orchestra.run.smithi019.stdout:Total 748 kB/s | 262 kB 00:00 2023-10-06T13:43:54.413 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:43:54.413 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:54.414 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:54.414 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:43:54.414 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:54.414 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:43:54.414 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 126 k 2023-10-06T13:43:54.414 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:54.414 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:43:54.414 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:54.414 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:43:54.414 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:54.414 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 126 k 2023-10-06T13:43:54.415 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 456 k 2023-10-06T13:43:54.415 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:43:54.421 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:43:54.421 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:43:54.456 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:43:54.456 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:43:54.682 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:43:54.731 INFO:teuthology.orchestra.run.smithi083.stdout:librados-devel-18.2.0-596.g4af54c4a.el9.x86_64. 400 kB/s | 126 kB 00:00 2023-10-06T13:43:54.731 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:54.732 INFO:teuthology.orchestra.run.smithi083.stdout:Total 398 kB/s | 126 kB 00:00 2023-10-06T13:43:54.732 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:43:54.738 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:43:54.738 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:43:54.791 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:43:54.792 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:43:54.864 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:18.2.0-596.g4af54c4a.el9.noarch 1/1 2023-10-06T13:43:54.880 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:18.2.0-596.g4af54c4a.el9.noarch 1/1 2023-10-06T13:43:54.880 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:43:54.880 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-06T13:43:54.880 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:54.932 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:43:55.037 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:43:55.474 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:43:55.794 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:43:55.795 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:55.795 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:43:55.795 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:55.795 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:55.795 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:55.951 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:18.2.0-596.g4af54c4a.el9.noarch 1/1 2023-10-06T13:43:55.951 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:55.952 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:43:55.952 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:18.2.0-596.g4af54c4a.el9.noarch 2023-10-06T13:43:55.952 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:55.952 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:43:56.011 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install libcephfs2 2023-10-06T13:43:56.276 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librados-devel 2023-10-06T13:43:56.624 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:29 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:56.713 INFO:teuthology.orchestra.run.smithi083.stdout:Package libcephfs2-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:43:56.762 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:56.762 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2023-10-06T13:43:56.763 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:56.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install libcephfs-devel 2023-10-06T13:43:56.900 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:28 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:43:57.044 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:43:57.045 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:57.045 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:43:57.045 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:57.046 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:43:57.046 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 126 k 2023-10-06T13:43:57.046 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:57.046 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:43:57.046 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:43:57.046 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:43:57.046 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:57.046 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 126 k 2023-10-06T13:43:57.046 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 456 k 2023-10-06T13:43:57.046 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:43:57.249 INFO:teuthology.orchestra.run.smithi019.stdout:librados-devel-18.2.0-596.g4af54c4a.el9.x86_64. 623 kB/s | 126 kB 00:00 2023-10-06T13:43:57.250 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:57.250 INFO:teuthology.orchestra.run.smithi019.stdout:Total 619 kB/s | 126 kB 00:00 2023-10-06T13:43:57.250 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:43:57.256 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:43:57.256 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:43:57.314 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:43:57.314 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:43:57.418 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:30 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:57.464 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:43:57.556 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:57.557 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:57.557 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:43:57.557 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:57.557 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:43:57.557 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 31 k 2023-10-06T13:43:57.557 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:57.557 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:43:57.558 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:43:57.558 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:43:57.558 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:57.558 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 31 k 2023-10-06T13:43:57.558 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 137 k 2023-10-06T13:43:57.558 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:43:57.559 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:43:57.793 INFO:teuthology.orchestra.run.smithi083.stdout:libcephfs-devel-18.2.0-596.g4af54c4a.el9.x86_64 130 kB/s | 31 kB 00:00 2023-10-06T13:43:57.794 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:43:57.794 INFO:teuthology.orchestra.run.smithi083.stdout:Total 129 kB/s | 31 kB 00:00 2023-10-06T13:43:57.795 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:43:57.797 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:43:57.797 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:43:57.828 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:43:57.828 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:43:57.917 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:43:58.018 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:43:58.024 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:43:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:43:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:58.360 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:43:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:43:58.360 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:43:58.431 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:43:58.601 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install libcephfs2 2023-10-06T13:43:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:43:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:58.702 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:43:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:43:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:43:58.702 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:58.899 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install librados2 2023-10-06T13:43:59.221 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:31 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:43:59.315 INFO:teuthology.orchestra.run.smithi019.stdout:Package libcephfs2-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:43:59.366 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:43:59.368 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-10-06T13:43:59.368 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:43:59.436 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install libcephfs-devel 2023-10-06T13:43:59.507 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:32 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:43:59.605 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:43:59.655 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:43:59.656 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2023-10-06T13:43:59.656 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:43:59.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install librbd1 2023-10-06T13:44:00.049 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:32 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:44:00.194 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:44:00.195 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:00.195 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:44:00.195 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:00.195 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:44:00.195 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 31 k 2023-10-06T13:44:00.196 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:00.196 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:44:00.196 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:00.196 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:44:00.196 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:00.196 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 31 k 2023-10-06T13:44:00.196 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 137 k 2023-10-06T13:44:00.196 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:44:00.322 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:33 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:44:00.412 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:44:00.444 INFO:teuthology.orchestra.run.smithi019.stdout:libcephfs-devel-18.2.0-596.g4af54c4a.el9.x86_64 123 kB/s | 31 kB 00:00 2023-10-06T13:44:00.445 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:44:00.445 INFO:teuthology.orchestra.run.smithi019.stdout:Total 123 kB/s | 31 kB 00:00 2023-10-06T13:44:00.445 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:44:00.448 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:44:00.448 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:44:00.461 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:44:00.462 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2023-10-06T13:44:00.463 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:44:00.483 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:44:00.483 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:44:00.536 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-rados 2023-10-06T13:44:00.577 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:44:00.703 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:01.081 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:01.128 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:34 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:44:01.216 INFO:teuthology.orchestra.run.smithi083.stdout:Package python3-rados-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:44:01.264 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:44:01.265 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2023-10-06T13:44:01.265 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:44:01.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-rgw 2023-10-06T13:44:01.368 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:01.368 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:01.368 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:44:01.369 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:44:01.369 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:01.369 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:44:01.572 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librados2 2023-10-06T13:44:01.940 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:34 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:44:02.029 INFO:teuthology.orchestra.run.smithi083.stdout:Package python3-rgw-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:44:02.078 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:44:02.079 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2023-10-06T13:44:02.079 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:44:02.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-cephfs 2023-10-06T13:44:02.193 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:34 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:44:02.285 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:44:02.338 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:44:02.339 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-10-06T13:44:02.339 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:44:02.402 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librbd1 2023-10-06T13:44:02.738 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:35 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:44:02.827 INFO:teuthology.orchestra.run.smithi083.stdout:Package python3-cephfs-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:44:02.876 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:44:02.877 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2023-10-06T13:44:02.877 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:44:02.948 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-rbd 2023-10-06T13:44:03.020 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:35 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:44:03.112 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:44:03.164 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:44:03.165 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-10-06T13:44:03.166 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:44:03.230 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rados 2023-10-06T13:44:03.541 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:36 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:44:03.629 INFO:teuthology.orchestra.run.smithi083.stdout:Package python3-rbd-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:44:03.678 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:44:03.679 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2023-10-06T13:44:03.679 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:44:03.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install rbd-fuse 2023-10-06T13:44:03.854 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:35 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:44:03.946 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rados-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:44:03.998 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:44:03.999 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-10-06T13:44:03.999 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:44:04.073 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rgw 2023-10-06T13:44:04.344 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:37 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:44:04.480 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:44:04.481 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2023-10-06T13:44:04.481 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:04.481 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:44:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 86 k 2023-10-06T13:44:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:04.481 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:44:04.481 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:04.481 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:44:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:04.482 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 86 k 2023-10-06T13:44:04.482 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 231 k 2023-10-06T13:44:04.482 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:44:04.697 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:36 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:44:04.697 INFO:teuthology.orchestra.run.smithi083.stdout:rbd-fuse-18.2.0-596.g4af54c4a.el9.x86_64.rpm 400 kB/s | 86 kB 00:00 2023-10-06T13:44:04.698 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:44:04.698 INFO:teuthology.orchestra.run.smithi083.stdout:Total 397 kB/s | 86 kB 00:00 2023-10-06T13:44:04.698 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:44:04.705 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:44:04.705 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:44:04.760 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:44:04.760 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:44:04.789 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rgw-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:44:04.841 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:44:04.842 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-10-06T13:44:04.842 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:44:04.893 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:44:04.910 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-cephfs 2023-10-06T13:44:04.986 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:05.389 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:05.527 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:37 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:44:05.620 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-cephfs-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:44:05.672 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:44:05.673 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-10-06T13:44:05.673 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:44:05.702 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:05.702 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:05.702 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:44:05.702 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:44:05.702 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:05.702 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:44:05.744 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rbd 2023-10-06T13:44:05.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install rbd-mirror 2023-10-06T13:44:06.364 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:38 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:44:06.456 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rbd-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:44:06.508 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:44:06.509 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-10-06T13:44:06.509 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:44:06.554 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:39 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:44:06.580 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-fuse 2023-10-06T13:44:06.694 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:44:06.695 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:44:06.695 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:06.695 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:44:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 3.0 M 2023-10-06T13:44:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:06.696 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:44:06.696 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:06.696 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:44:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:06.696 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 3.0 M 2023-10-06T13:44:06.696 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 12 M 2023-10-06T13:44:06.696 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:44:07.109 INFO:teuthology.orchestra.run.smithi083.stdout:rbd-mirror-18.2.0-596.g4af54c4a.el9.x86_64.rpm 7.2 MB/s | 3.0 MB 00:00 2023-10-06T13:44:07.110 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:44:07.110 INFO:teuthology.orchestra.run.smithi083.stdout:Total 7.2 MB/s | 3.0 MB 00:00 2023-10-06T13:44:07.110 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:44:07.119 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:44:07.119 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:44:07.190 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:44:07.190 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:44:07.210 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:39 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:44:07.357 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:44:07.358 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:07.358 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2023-10-06T13:44:07.359 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:07.359 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:44:07.359 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 86 k 2023-10-06T13:44:07.359 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:07.359 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:44:07.359 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:07.360 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:44:07.360 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:07.360 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 86 k 2023-10-06T13:44:07.360 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 231 k 2023-10-06T13:44:07.360 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:44:07.547 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-fuse-18.2.0-596.g4af54c4a.el9.x86_64.rpm 458 kB/s | 86 kB 00:00 2023-10-06T13:44:07.548 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:44:07.548 INFO:teuthology.orchestra.run.smithi019.stdout:Total 454 kB/s | 86 kB 00:00 2023-10-06T13:44:07.548 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:44:07.555 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:44:07.555 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:44:07.608 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:44:07.614 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:44:07.614 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:44:07.659 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:07.684 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:07.684 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:44:07.685 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-06T13:44:07.685 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-06T13:44:07.685 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-06T13:44:07.685 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:07.758 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:44:07.856 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:08.235 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:08.566 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:08.566 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:08.566 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:44:08.566 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:44:08.567 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:08.567 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:44:08.806 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-mirror 2023-10-06T13:44:08.859 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:08.859 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:08.859 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:44:08.860 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:44:08.860 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:08.860 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:44:09.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install rbd-nbd 2023-10-06T13:44:09.427 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:41 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:44:09.573 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:44:09.573 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:09.574 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:44:09.574 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:09.574 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:44:09.574 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 3.0 M 2023-10-06T13:44:09.574 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:09.574 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:44:09.574 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:09.574 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:44:09.574 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:09.574 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 3.0 M 2023-10-06T13:44:09.574 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 12 M 2023-10-06T13:44:09.575 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:44:09.725 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:42 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:44:09.863 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:44:09.864 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:09.864 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2023-10-06T13:44:09.864 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:09.864 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:44:09.864 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 170 k 2023-10-06T13:44:09.865 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:09.865 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:44:09.865 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:09.865 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:44:09.865 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:09.865 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 170 k 2023-10-06T13:44:09.865 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 490 k 2023-10-06T13:44:09.865 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:44:10.017 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-mirror-18.2.0-596.g4af54c4a.el9.x86_64.rpm 6.7 MB/s | 3.0 MB 00:00 2023-10-06T13:44:10.018 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:44:10.018 INFO:teuthology.orchestra.run.smithi019.stdout:Total 6.7 MB/s | 3.0 MB 00:00 2023-10-06T13:44:10.018 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:44:10.028 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:44:10.028 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:44:10.071 INFO:teuthology.orchestra.run.smithi083.stdout:rbd-nbd-18.2.0-596.g4af54c4a.el9.x86_64.rpm 822 kB/s | 170 kB 00:00 2023-10-06T13:44:10.072 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:44:10.072 INFO:teuthology.orchestra.run.smithi083.stdout:Total 817 kB/s | 170 kB 00:00 2023-10-06T13:44:10.072 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:44:10.080 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:44:10.080 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:44:10.104 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:44:10.105 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:44:10.135 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:44:10.135 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:44:10.276 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:44:10.369 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:10.550 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:44:10.618 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:10.643 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-06T13:44:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-06T13:44:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-06T13:44:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-06T13:44:10.644 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:10.816 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-nbd-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:11.110 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:11.110 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:11.110 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:44:11.110 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:44:11.110 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:11.111 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:44:11.381 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-base-debuginfo 2023-10-06T13:44:11.757 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:11.757 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:11.757 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:44:11.758 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:44:11.758 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:44:11.990 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:44 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:44:12.008 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-nbd 2023-10-06T13:44:12.128 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:44:12.129 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:44:12.129 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:12.129 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:44:12.129 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 74 M 2023-10-06T13:44:12.129 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2023-10-06T13:44:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 20 M 2023-10-06T13:44:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:12.130 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:44:12.130 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:12.130 INFO:teuthology.orchestra.run.smithi083.stdout:Install 2 Packages 2023-10-06T13:44:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:12.130 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 94 M 2023-10-06T13:44:12.131 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 512 M 2023-10-06T13:44:12.131 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:44:12.635 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:44 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:44:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:44:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2023-10-06T13:44:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:44:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 170 k 2023-10-06T13:44:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:44:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:44:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 170 k 2023-10-06T13:44:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 490 k 2023-10-06T13:44:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:44:13.312 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-nbd-18.2.0-596.g4af54c4a.el9.x86_64.rpm 319 kB/s | 170 kB 00:00 2023-10-06T13:44:13.312 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:44:13.313 INFO:teuthology.orchestra.run.smithi019.stdout:Total 318 kB/s | 170 kB 00:00 2023-10-06T13:44:13.313 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:44:13.322 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:44:13.322 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:44:13.385 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:44:13.385 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:44:13.535 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:44:13.630 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:14.054 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-nbd-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:14.359 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:44:14.359 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:14.359 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:44:14.359 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:44:14.359 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:14.359 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:44:14.590 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-base-debuginfo 2023-10-06T13:44:14.917 INFO:teuthology.orchestra.run.smithi083.stdout:(1/2): ceph-debuginfo-18.2.0-596.g4af54c4a.el9. 7.4 MB/s | 20 MB 00:02 2023-10-06T13:44:15.230 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:47 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:44:15.378 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:44:15.378 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:44:15.378 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:15.378 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:44:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 74 M 2023-10-06T13:44:15.379 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-10-06T13:44:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 20 M 2023-10-06T13:44:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:15.379 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:44:15.379 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:15.379 INFO:teuthology.orchestra.run.smithi019.stdout:Install 2 Packages 2023-10-06T13:44:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:15.380 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 94 M 2023-10-06T13:44:15.380 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 512 M 2023-10-06T13:44:15.380 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:44:16.367 INFO:teuthology.orchestra.run.smithi083.stdout:(2/2): ceph-base-debuginfo-18.2.0-596.g4af54c4a 17 MB/s | 74 MB 00:04 2023-10-06T13:44:16.369 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:44:16.369 INFO:teuthology.orchestra.run.smithi083.stdout:Total 22 MB/s | 94 MB 00:04 2023-10-06T13:44:16.370 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:44:16.374 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:44:16.374 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:44:16.983 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:44:16.983 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:44:17.290 INFO:teuthology.orchestra.run.smithi019.stdout:(1/2): ceph-debuginfo-18.2.0-596.g4af54c4a.el9. 11 MB/s | 20 MB 00:01 2023-10-06T13:44:19.913 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:44:20.523 INFO:teuthology.orchestra.run.smithi019.stdout:(2/2): ceph-base-debuginfo-18.2.0-596.g4af54c4a 14 MB/s | 74 MB 00:05 2023-10-06T13:44:20.526 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:44:20.526 INFO:teuthology.orchestra.run.smithi019.stdout:Total 18 MB/s | 94 MB 00:05 2023-10-06T13:44:20.526 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:44:20.531 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:44:20.531 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:44:21.174 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:44:21.174 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:44:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:44:27.545 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/2 2023-10-06T13:44:27.618 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 2/2 2023-10-06T13:44:31.228 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 2/2 2023-10-06T13:44:31.229 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/2 2023-10-06T13:44:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2/2 2023-10-06T13:44:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:31.505 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:44:31.506 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:44:31.506 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:44:31.506 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:31.506 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:44:31.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-common-debuginfo 2023-10-06T13:44:32.313 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:02:05 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:44:32.377 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/2 2023-10-06T13:44:32.450 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:44:32.451 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:32.451 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:44:32.452 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:32.452 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:44:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 404 M 2023-10-06T13:44:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:32.452 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:44:32.452 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:44:32.452 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:44:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:44:32.453 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 404 M 2023-10-06T13:44:32.453 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 1.7 G 2023-10-06T13:44:32.453 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:44:32.454 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 2/2 2023-10-06T13:44:35.846 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 2/2 2023-10-06T13:44:35.847 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/2 2023-10-06T13:44:36.137 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2/2 2023-10-06T13:44:36.137 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:36.138 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:44:36.138 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:44:36.138 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:44:36.138 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:36.138 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:44:36.383 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-common-debuginfo 2023-10-06T13:44:37.008 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:09 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:44:37.153 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:44:37.153 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:44:37.153 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:37.153 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:44:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 404 M 2023-10-06T13:44:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:37.153 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:44:37.153 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:44:37.154 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:44:37.154 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:44:37.154 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 404 M 2023-10-06T13:44:37.154 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 1.7 G 2023-10-06T13:44:37.154 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:44:51.995 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-common-debuginfo-18.2.0-596.g4af54c4a.el9. 21 MB/s | 404 MB 00:19 2023-10-06T13:44:51.996 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:44:51.997 INFO:teuthology.orchestra.run.smithi083.stdout:Total 21 MB/s | 404 MB 00:19 2023-10-06T13:44:51.997 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:44:52.001 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:44:52.001 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:44:54.415 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:44:54.416 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:45:05.000 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-common-debuginfo-18.2.0-596.g4af54c4a.el9. 15 MB/s | 404 MB 00:27 2023-10-06T13:45:05.001 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:45:05.002 INFO:teuthology.orchestra.run.smithi019.stdout:Total 15 MB/s | 404 MB 00:27 2023-10-06T13:45:05.002 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:45:05.006 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:45:05.006 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:45:07.541 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:45:07.541 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:45:35.799 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:45:35.890 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:45:44.054 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:45:44.335 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:45:44.335 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:45:44.335 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:45:44.335 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:45:44.335 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:45:44.335 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:45:44.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2023-10-06T13:45:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:03:18 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:45:45.322 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:45:45.322 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2023-10-06T13:45:45.323 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:45:45.323 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2023-10-06T13:45:45.323 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:45:45.323 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 1.4 M 2023-10-06T13:45:45.323 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:45:45.323 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:45:45.323 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2023-10-06T13:45:45.323 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:45:45.323 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:45:45.323 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 1.4 M 2023-10-06T13:45:45.323 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 6.2 M 2023-10-06T13:45:45.324 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:45:45.996 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-immutable-object-cache-debuginfo-18.2.0-59 2.1 MB/s | 1.4 MB 00:00 2023-10-06T13:45:45.996 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:45:45.997 INFO:teuthology.orchestra.run.smithi083.stdout:Total 2.1 MB/s | 1.4 MB 00:00 2023-10-06T13:45:45.997 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:45:46.000 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:45:46.000 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:45:46.042 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:45:46.043 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:45:46.298 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:45:46.402 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:18.2.0-596.g 1/1 2023-10-06T13:45:46.828 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:18.2.0-596.g 1/1 2023-10-06T13:45:47.111 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:18.2.0-596.g 1/1 2023-10-06T13:45:47.112 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:45:47.112 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:45:47.112 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:45:47.112 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:45:47.112 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:45:47.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw-debuginfo 2023-10-06T13:45:47.867 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:03:20 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:45:48.008 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:45:48.008 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:45:48.008 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:45:48.008 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:45:48.008 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:45:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 162 M 2023-10-06T13:45:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:45:48.009 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:45:48.009 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:45:48.009 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:45:48.009 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:45:48.010 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 162 M 2023-10-06T13:45:48.010 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 678 M 2023-10-06T13:45:48.010 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:45:51.669 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:45:51.891 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:45:55.423 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-radosgw-debuginfo-18.2.0-596.g4af54c4a.el9 22 MB/s | 162 MB 00:07 2023-10-06T13:45:55.424 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:45:55.425 INFO:teuthology.orchestra.run.smithi083.stdout:Total 22 MB/s | 162 MB 00:07 2023-10-06T13:45:55.425 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:45:55.429 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:45:55.429 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:45:56.423 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:45:56.423 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:45:59.741 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:46:00.068 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:46:00.069 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:46:00.069 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:46:00.069 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:46:00.069 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:46:00.070 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:46:00.297 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2023-10-06T13:46:00.927 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:03:32 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:46:01.074 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:46:01.075 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2023-10-06T13:46:01.075 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:46:01.075 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2023-10-06T13:46:01.075 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:46:01.075 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 1.4 M 2023-10-06T13:46:01.076 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:46:01.076 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:46:01.076 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2023-10-06T13:46:01.076 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:46:01.076 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:46:01.076 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 1.4 M 2023-10-06T13:46:01.076 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 6.2 M 2023-10-06T13:46:01.076 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:46:01.517 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-immutable-object-cache-debuginfo-18.2.0-59 3.2 MB/s | 1.4 MB 00:00 2023-10-06T13:46:01.517 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:46:01.518 INFO:teuthology.orchestra.run.smithi019.stdout:Total 3.2 MB/s | 1.4 MB 00:00 2023-10-06T13:46:01.518 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:46:01.522 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:46:01.522 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:46:01.566 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:46:01.566 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:46:01.832 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:46:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:18.2.0-596.g 1/1 2023-10-06T13:46:02.361 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:18.2.0-596.g 1/1 2023-10-06T13:46:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:18.2.0-596.g 1/1 2023-10-06T13:46:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:46:02.645 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:46:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:46:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:46:02.645 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:46:02.807 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw-debuginfo 2023-10-06T13:46:03.427 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:03:35 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:46:03.571 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:46:03.571 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:46:03.572 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:46:03.572 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:46:03.572 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:46:03.572 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 162 M 2023-10-06T13:46:03.572 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:46:03.572 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:46:03.572 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:46:03.572 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:46:03.572 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:46:03.572 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 162 M 2023-10-06T13:46:03.572 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 678 M 2023-10-06T13:46:03.573 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:46:10.954 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-radosgw-debuginfo-18.2.0-596.g4af54c4a.el9 22 MB/s | 162 MB 00:07 2023-10-06T13:46:10.954 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:46:10.955 INFO:teuthology.orchestra.run.smithi019.stdout:Total 22 MB/s | 162 MB 00:07 2023-10-06T13:46:10.955 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:46:10.959 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:46:10.959 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:46:11.995 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:46:11.995 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:46:12.905 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:46:12.995 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x8 1/1 2023-10-06T13:46:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x8 1/1 2023-10-06T13:46:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x8 1/1 2023-10-06T13:46:17.607 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:46:17.608 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:46:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:46:17.608 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:46:17.608 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:46:17.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-test-debuginfo 2023-10-06T13:46:18.387 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:03:51 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:46:18.523 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:46:18.524 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:46:18.524 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:46:18.524 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:46:18.524 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:46:18.525 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 601 M 2023-10-06T13:46:18.525 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:46:18.525 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:46:18.525 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:46:18.525 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:46:18.525 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:46:18.525 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 601 M 2023-10-06T13:46:18.525 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 2.7 G 2023-10-06T13:46:18.525 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:46:29.658 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:46:29.751 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x8 1/1 2023-10-06T13:46:33.978 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x8 1/1 2023-10-06T13:46:34.254 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x8 1/1 2023-10-06T13:46:34.254 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:46:34.254 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:46:34.254 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:46:34.254 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:46:34.254 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:46:34.432 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-test-debuginfo 2023-10-06T13:46:35.054 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:04:07 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:46:35.198 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:46:35.199 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:46:35.199 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:46:35.199 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:46:35.199 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:46:35.199 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 601 M 2023-10-06T13:46:35.199 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:46:35.199 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:46:35.199 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:46:35.200 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:46:35.200 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:46:35.200 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 601 M 2023-10-06T13:46:35.200 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 2.7 G 2023-10-06T13:46:35.200 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:46:46.097 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-test-debuginfo-18.2.0-596.g4af54c4a.el9.x8 22 MB/s | 601 MB 00:27 2023-10-06T13:46:46.098 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:46:46.098 INFO:teuthology.orchestra.run.smithi083.stdout:Total 22 MB/s | 601 MB 00:27 2023-10-06T13:46:46.099 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:46:46.104 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:46:46.104 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:46:49.661 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:46:49.662 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:47:05.443 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-test-debuginfo-18.2.0-596.g4af54c4a.el9.x8 20 MB/s | 601 MB 00:30 2023-10-06T13:47:05.444 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:47:05.444 INFO:teuthology.orchestra.run.smithi019.stdout:Total 20 MB/s | 601 MB 00:30 2023-10-06T13:47:05.444 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:47:05.450 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:47:05.450 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:47:09.197 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:47:09.197 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:47:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:47:53.073 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:48:01.649 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-test-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:48:01.918 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:48:01.918 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:01.918 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:48:01.918 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:48:01.918 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:01.918 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:48:02.196 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-base-debuginfo 2023-10-06T13:48:02.789 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:05:35 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:48:02.878 INFO:teuthology.orchestra.run.smithi083.stdout:Package ceph-base-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:48:02.927 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:48:02.928 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2023-10-06T13:48:02.928 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:48:03.006 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr-debuginfo 2023-10-06T13:48:03.596 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:05:36 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:48:03.732 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:48:03.733 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:48:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:48:03.734 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:48:03.734 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:48:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 19 M 2023-10-06T13:48:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:03.735 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:48:03.735 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:48:03.735 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:48:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:03.735 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 19 M 2023-10-06T13:48:03.735 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 101 M 2023-10-06T13:48:03.735 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:48:05.036 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-mgr-debuginfo-18.2.0-596.g4af54c4a.el9.x86 15 MB/s | 19 MB 00:01 2023-10-06T13:48:05.036 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:48:05.037 INFO:teuthology.orchestra.run.smithi083.stdout:Total 15 MB/s | 19 MB 00:01 2023-10-06T13:48:05.037 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:48:05.040 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:48:05.041 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:48:05.185 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:48:05.185 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:48:07.356 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:48:07.526 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:08.559 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:08.841 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:08.842 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:08.842 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:48:08.842 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:48:08.842 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:08.842 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:48:09.032 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mds-debuginfo 2023-10-06T13:48:09.623 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:05:42 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:48:09.762 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:48:09.762 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:48:09.762 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:48:09.762 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:48:09.762 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:48:09.762 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 48 M 2023-10-06T13:48:09.763 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:09.763 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:48:09.763 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:48:09.763 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:48:09.763 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:09.763 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 48 M 2023-10-06T13:48:09.763 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 203 M 2023-10-06T13:48:09.764 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:48:12.097 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-mds-debuginfo-18.2.0-596.g4af54c4a.el9.x86 21 MB/s | 48 MB 00:02 2023-10-06T13:48:12.098 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:48:12.098 INFO:teuthology.orchestra.run.smithi083.stdout:Total 21 MB/s | 48 MB 00:02 2023-10-06T13:48:12.099 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:48:12.103 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:48:12.103 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:48:12.420 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:48:12.420 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:48:15.843 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:48:16.405 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:48:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:48:17.460 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:19.034 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:19.323 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:19.323 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:19.323 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:48:19.323 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:48:19.324 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:19.324 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:48:19.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mon-debuginfo 2023-10-06T13:48:20.094 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:05:53 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:48:20.231 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:48:20.232 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:48:20.232 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:48:20.232 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:48:20.232 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:48:20.232 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 113 M 2023-10-06T13:48:20.233 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:20.233 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:48:20.233 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:48:20.233 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:48:20.233 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:20.234 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 113 M 2023-10-06T13:48:20.234 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 525 M 2023-10-06T13:48:20.234 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:48:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-test-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:48:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:48:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:48:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:48:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:48:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:48:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:48:24.047 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-base-debuginfo 2023-10-06T13:48:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:05:56 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:48:24.761 INFO:teuthology.orchestra.run.smithi019.stdout:Package ceph-base-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 is already installed. 2023-10-06T13:48:24.813 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:48:24.814 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-10-06T13:48:24.814 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:48:24.885 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-debuginfo 2023-10-06T13:48:25.508 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:05:57 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:48:25.532 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-mon-debuginfo-18.2.0-596.g4af54c4a.el9.x86 21 MB/s | 113 MB 00:05 2023-10-06T13:48:25.533 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:48:25.534 INFO:teuthology.orchestra.run.smithi083.stdout:Total 21 MB/s | 113 MB 00:05 2023-10-06T13:48:25.534 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:48:25.538 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:48:25.538 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:48:25.653 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:48:25.654 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:48:25.654 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:48:25.654 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:48:25.654 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:48:25.654 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 19 M 2023-10-06T13:48:25.655 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:48:25.655 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:48:25.655 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:48:25.655 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:48:25.655 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:48:25.655 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 19 M 2023-10-06T13:48:25.655 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 101 M 2023-10-06T13:48:25.655 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:48:26.235 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:48:26.235 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:48:26.873 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-mgr-debuginfo-18.2.0-596.g4af54c4a.el9.x86 16 MB/s | 19 MB 00:01 2023-10-06T13:48:26.873 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:48:26.874 INFO:teuthology.orchestra.run.smithi019.stdout:Total 16 MB/s | 19 MB 00:01 2023-10-06T13:48:26.874 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:48:26.878 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:48:26.878 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:48:27.031 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:48:27.031 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:48:29.389 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:48:29.482 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:30.561 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:30.839 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:30.839 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:48:30.840 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:48:30.840 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:48:30.840 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:48:30.840 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:48:31.111 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mds-debuginfo 2023-10-06T13:48:31.731 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:06:03 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:48:31.875 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:48:31.876 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:48:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:48:31.876 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:48:31.876 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:48:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 48 M 2023-10-06T13:48:31.876 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:48:31.876 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:48:31.877 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:48:31.877 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:48:31.877 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:48:31.877 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 48 M 2023-10-06T13:48:31.877 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 203 M 2023-10-06T13:48:31.877 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:48:34.375 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-mds-debuginfo-18.2.0-596.g4af54c4a.el9.x86 19 MB/s | 48 MB 00:02 2023-10-06T13:48:34.376 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:48:34.377 INFO:teuthology.orchestra.run.smithi019.stdout:Total 19 MB/s | 48 MB 00:02 2023-10-06T13:48:34.377 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:48:34.381 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:48:34.381 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:48:34.717 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:48:34.717 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:48:38.039 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:48:38.214 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:48:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:41.683 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:41.686 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:41.963 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:41.963 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:48:41.963 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:48:41.964 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:48:41.964 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:48:41.964 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:48:41.964 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:48:41.964 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:41.965 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:48:41.965 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:48:41.965 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:41.965 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:48:42.142 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mon-debuginfo 2023-10-06T13:48:42.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-osd-debuginfo 2023-10-06T13:48:42.732 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:06:15 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:48:42.765 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:06:14 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:48:42.869 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:48:42.870 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:48:42.871 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:48:42.871 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:48:42.871 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:48:42.871 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 312 M 2023-10-06T13:48:42.871 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:42.871 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:48:42.871 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:48:42.871 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:48:42.871 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:48:42.871 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 312 M 2023-10-06T13:48:42.872 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 1.3 G 2023-10-06T13:48:42.872 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:48:42.911 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:48:42.912 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:48:42.913 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:48:42.913 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:48:42.913 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:48:42.913 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 113 M 2023-10-06T13:48:42.913 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:48:42.913 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:48:42.913 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:48:42.913 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:48:42.913 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:48:42.914 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 113 M 2023-10-06T13:48:42.914 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 525 M 2023-10-06T13:48:42.914 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:48:56.983 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-osd-debuginfo-18.2.0-596.g4af54c4a.el9.x86 22 MB/s | 312 MB 00:14 2023-10-06T13:48:56.984 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:48:56.984 INFO:teuthology.orchestra.run.smithi083.stdout:Total 22 MB/s | 312 MB 00:14 2023-10-06T13:48:56.985 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:48:56.989 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:48:56.989 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:48:58.848 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:48:58.848 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:49:01.242 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-mon-debuginfo-18.2.0-596.g4af54c4a.el9.x86 6.1 MB/s | 113 MB 00:18 2023-10-06T13:49:01.340 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:49:01.341 INFO:teuthology.orchestra.run.smithi019.stdout:Total 6.1 MB/s | 113 MB 00:18 2023-10-06T13:49:01.341 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:49:01.341 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:49:01.341 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:49:01.978 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:49:01.979 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:49:14.550 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:49:14.643 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:49:17.996 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:49:18.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:49:18.286 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:49:18.286 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:49:18.286 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:49:18.286 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:49:18.286 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:49:18.474 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-osd-debuginfo 2023-10-06T13:49:19.096 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:06:51 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:49:19.243 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:49:19.244 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:49:19.244 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:49:19.244 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:49:19.244 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:49:19.244 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 312 M 2023-10-06T13:49:19.244 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:49:19.244 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:49:19.244 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:49:19.244 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:49:19.245 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:49:19.245 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 312 M 2023-10-06T13:49:19.245 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 1.3 G 2023-10-06T13:49:19.245 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:49:30.650 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:49:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:49:32.834 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-osd-debuginfo-18.2.0-596.g4af54c4a.el9.x86 23 MB/s | 312 MB 00:13 2023-10-06T13:49:32.835 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:49:32.835 INFO:teuthology.orchestra.run.smithi019.stdout:Total 23 MB/s | 312 MB 00:13 2023-10-06T13:49:32.835 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:49:32.840 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:49:32.840 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:49:34.799 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:49:34.799 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:49:39.075 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:49:39.353 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:49:39.353 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:39.353 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:49:39.354 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:49:39.354 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:39.354 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:49:39.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-fuse-debuginfo 2023-10-06T13:49:40.146 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:07:13 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:49:40.283 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:49:40.284 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:49:40.284 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:40.284 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:49:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 5.7 M 2023-10-06T13:49:40.284 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:40.284 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:49:40.285 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:40.285 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:49:40.285 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:40.285 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 5.7 M 2023-10-06T13:49:40.285 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 27 M 2023-10-06T13:49:40.285 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:49:40.774 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-fuse-debuginfo-18.2.0-596.g4af54c4a.el9.x8 12 MB/s | 5.7 MB 00:00 2023-10-06T13:49:40.775 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:49:40.776 INFO:teuthology.orchestra.run.smithi083.stdout:Total 12 MB/s | 5.7 MB 00:00 2023-10-06T13:49:40.776 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:49:40.779 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:49:40.779 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:49:40.845 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:49:40.845 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:49:41.594 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:49:41.706 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:49:42.280 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:49:42.555 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:49:42.556 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:42.556 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:49:42.556 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:49:42.556 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:42.556 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:49:42.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install librados-devel-debuginfo 2023-10-06T13:49:43.310 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:07:16 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:49:43.446 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:49:43.447 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:43.447 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:49:43.447 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:43.447 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:49:43.447 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 108 k 2023-10-06T13:49:43.447 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:43.447 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:49:43.448 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:43.448 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:49:43.448 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:43.448 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 108 k 2023-10-06T13:49:43.448 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 440 k 2023-10-06T13:49:43.448 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:49:43.811 INFO:teuthology.orchestra.run.smithi083.stdout:librados-devel-debuginfo-18.2.0-596.g4af54c4a.e 298 kB/s | 108 kB 00:00 2023-10-06T13:49:43.811 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:49:43.812 INFO:teuthology.orchestra.run.smithi083.stdout:Total 297 kB/s | 108 kB 00:00 2023-10-06T13:49:43.812 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:49:43.815 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:49:43.815 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:49:43.847 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:49:43.847 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:49:43.968 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:49:44.076 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-debuginfo-2:18.2.0-596.g4af54c4a.el9. 1/1 2023-10-06T13:49:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados-devel-debuginfo-2:18.2.0-596.g4af54c4a.el9. 1/1 2023-10-06T13:49:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-debuginfo-2:18.2.0-596.g4af54c4a.el9. 1/1 2023-10-06T13:49:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:44.779 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:49:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:49:44.779 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:44.779 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:49:44.980 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install libcephfs2-debuginfo 2023-10-06T13:49:45.567 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:07:18 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:49:45.705 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:49:45.706 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:49:45.706 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:45.706 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:49:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 4.8 M 2023-10-06T13:49:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:45.707 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:49:45.707 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:45.707 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:49:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:45.707 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 4.8 M 2023-10-06T13:49:45.707 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 23 M 2023-10-06T13:49:45.707 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:49:46.156 INFO:teuthology.orchestra.run.smithi083.stdout:libcephfs2-debuginfo-18.2.0-596.g4af54c4a.el9.x 11 MB/s | 4.8 MB 00:00 2023-10-06T13:49:46.156 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:49:46.157 INFO:teuthology.orchestra.run.smithi083.stdout:Total 11 MB/s | 4.8 MB 00:00 2023-10-06T13:49:46.157 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:49:46.161 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:49:46.161 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:49:46.220 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:49:46.221 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:49:46.829 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:49:46.929 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_ 1/1 2023-10-06T13:49:47.446 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_ 1/1 2023-10-06T13:49:47.711 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_ 1/1 2023-10-06T13:49:47.711 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:47.711 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:49:47.711 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:49:47.711 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:47.711 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:49:47.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install librados2-debuginfo 2023-10-06T13:49:48.477 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:07:21 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:49:48.613 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:49:48.614 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:48.614 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:49:48.614 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:48.614 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:49:48.615 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 44 M 2023-10-06T13:49:48.615 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:48.615 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:49:48.615 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:48.615 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:49:48.615 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:48.615 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 44 M 2023-10-06T13:49:48.615 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 238 M 2023-10-06T13:49:48.615 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:49:50.875 INFO:teuthology.orchestra.run.smithi083.stdout:librados2-debuginfo-18.2.0-596.g4af54c4a.el9.x8 20 MB/s | 44 MB 00:02 2023-10-06T13:49:50.876 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:49:50.877 INFO:teuthology.orchestra.run.smithi083.stdout:Total 19 MB/s | 44 MB 00:02 2023-10-06T13:49:50.877 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:49:50.881 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:49:50.881 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:49:51.183 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:49:51.183 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:49:56.125 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:49:56.222 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:49:58.045 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:49:58.334 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:49:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:58.335 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:49:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:49:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:58.335 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:49:58.537 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install librbd1-debuginfo 2023-10-06T13:49:59.120 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:07:32 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:49:59.257 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:49:59.258 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:49:59.258 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:59.258 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:49:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 51 M 2023-10-06T13:49:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:59.258 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:49:59.258 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:49:59.258 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:49:59.259 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:49:59.259 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 51 M 2023-10-06T13:49:59.259 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 292 M 2023-10-06T13:49:59.259 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:50:01.936 INFO:teuthology.orchestra.run.smithi083.stdout:librbd1-debuginfo-18.2.0-596.g4af54c4a.el9.x86_ 19 MB/s | 51 MB 00:02 2023-10-06T13:50:01.937 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:01.937 INFO:teuthology.orchestra.run.smithi083.stdout:Total 19 MB/s | 51 MB 00:02 2023-10-06T13:50:01.938 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:50:01.942 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:50:01.942 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:50:02.280 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:50:02.281 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:50:08.093 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:50:08.185 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librbd1-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:50:08.716 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:10.414 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:10.693 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:10.693 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:10.693 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:50:10.693 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:10.693 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:10.693 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:50:10.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-cephfs-debuginfo 2023-10-06T13:50:11.468 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:07:44 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:50:11.605 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:50:11.605 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:11.605 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:50:11.606 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:11.606 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:50:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 593 k 2023-10-06T13:50:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:11.606 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:50:11.606 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:11.606 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:50:11.606 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:11.607 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 593 k 2023-10-06T13:50:11.607 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 2.0 M 2023-10-06T13:50:11.607 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:50:11.873 INFO:teuthology.orchestra.run.smithi083.stdout:python3-cephfs-debuginfo-18.2.0-596.g4af54c4a.e 2.2 MB/s | 593 kB 00:00 2023-10-06T13:50:11.876 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:11.876 INFO:teuthology.orchestra.run.smithi083.stdout:Total 2.2 MB/s | 593 kB 00:00 2023-10-06T13:50:11.877 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:50:11.877 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:50:11.877 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:50:11.913 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:50:11.913 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:50:12.072 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:50:12.182 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-debuginfo-2:18.2.0-596.g4af54c4a.el9. 1/1 2023-10-06T13:50:12.580 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: python3-cephfs-debuginfo-2:18.2.0-596.g4af54c4a.el9. 1/1 2023-10-06T13:50:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-debuginfo-2:18.2.0-596.g4af54c4a.el9. 1/1 2023-10-06T13:50:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:12.860 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:50:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:12.860 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:50:13.055 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-rados-debuginfo 2023-10-06T13:50:13.644 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:07:46 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:50:13.780 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:50:13.780 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:13.781 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:50:13.781 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:13.781 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:50:13.781 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 770 k 2023-10-06T13:50:13.781 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:13.781 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:50:13.781 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:13.781 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:50:13.781 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:13.782 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 770 k 2023-10-06T13:50:13.782 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 2.7 M 2023-10-06T13:50:13.782 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:50:14.051 INFO:teuthology.orchestra.run.smithi083.stdout:python3-rados-debuginfo-18.2.0-596.g4af54c4a.el 2.8 MB/s | 770 kB 00:00 2023-10-06T13:50:14.051 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:14.051 INFO:teuthology.orchestra.run.smithi083.stdout:Total 2.8 MB/s | 770 kB 00:00 2023-10-06T13:50:14.052 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:50:14.055 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:50:14.055 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:50:14.090 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:50:14.091 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:50:14.269 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:50:14.373 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-debuginfo-2:18.2.0-596.g4af54c4a.el9.x 1/1 2023-10-06T13:50:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: python3-rados-debuginfo-2:18.2.0-596.g4af54c4a.el9.x 1/1 2023-10-06T13:50:15.032 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-debuginfo-2:18.2.0-596.g4af54c4a.el9.x 1/1 2023-10-06T13:50:15.032 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:15.032 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:50:15.032 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:15.033 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:15.033 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:50:15.222 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-rbd-debuginfo 2023-10-06T13:50:15.811 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:07:48 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:50:15.950 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:50:15.950 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:15.950 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:50:15.950 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:15.950 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:50:15.950 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 1.1 M 2023-10-06T13:50:15.950 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:15.950 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:50:15.951 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:15.951 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:50:15.951 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:15.951 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 1.1 M 2023-10-06T13:50:15.951 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 4.3 M 2023-10-06T13:50:15.951 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:50:16.237 INFO:teuthology.orchestra.run.smithi083.stdout:python3-rbd-debuginfo-18.2.0-596.g4af54c4a.el9. 3.9 MB/s | 1.1 MB 00:00 2023-10-06T13:50:16.238 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:16.238 INFO:teuthology.orchestra.run.smithi083.stdout:Total 3.9 MB/s | 1.1 MB 00:00 2023-10-06T13:50:16.238 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:50:16.241 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:50:16.241 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:50:16.281 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:50:16.281 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:50:16.502 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:50:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:50:16.771 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:17.040 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: python3-rbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:50:17.061 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:17.061 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:17.062 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:50:17.062 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:17.062 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:17.062 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:50:17.295 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-fuse-debuginfo 2023-10-06T13:50:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:50:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:17.329 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:50:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:17.329 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:50:17.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-rgw-debuginfo 2023-10-06T13:50:17.918 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:07:49 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:50:18.064 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:50:18.064 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:18.065 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:50:18.065 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:18.065 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:50:18.065 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 5.7 M 2023-10-06T13:50:18.065 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:18.065 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:50:18.065 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:18.065 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:50:18.065 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:18.065 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 5.7 M 2023-10-06T13:50:18.065 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 27 M 2023-10-06T13:50:18.066 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:50:18.118 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:07:51 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:50:18.255 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:50:18.255 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:18.256 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:50:18.256 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:18.256 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:50:18.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 337 k 2023-10-06T13:50:18.256 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:18.256 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:50:18.256 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:18.256 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:50:18.256 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:18.256 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 337 k 2023-10-06T13:50:18.257 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 1.0 M 2023-10-06T13:50:18.257 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:50:18.524 INFO:teuthology.orchestra.run.smithi083.stdout:python3-rgw-debuginfo-18.2.0-596.g4af54c4a.el9. 1.2 MB/s | 337 kB 00:00 2023-10-06T13:50:18.524 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:18.525 INFO:teuthology.orchestra.run.smithi083.stdout:Total 1.2 MB/s | 337 kB 00:00 2023-10-06T13:50:18.525 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:50:18.528 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:50:18.528 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:50:18.562 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:50:18.563 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:50:18.694 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:50:18.717 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-fuse-debuginfo-18.2.0-596.g4af54c4a.el9.x8 8.8 MB/s | 5.7 MB 00:00 2023-10-06T13:50:18.718 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:18.718 INFO:teuthology.orchestra.run.smithi019.stdout:Total 8.7 MB/s | 5.7 MB 00:00 2023-10-06T13:50:18.718 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:50:18.722 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:50:18.722 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:50:18.793 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:50:18.793 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:50:18.796 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:50:19.155 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: python3-rgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:50:19.433 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:50:19.433 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:19.433 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:50:19.433 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:19.433 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:19.433 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:50:19.587 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:50:19.660 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install rbd-fuse-debuginfo 2023-10-06T13:50:19.702 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:50:20.241 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:50:20.249 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:07:53 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:50:20.387 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:50:20.388 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:20.388 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:50:20.388 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:20.388 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:50:20.388 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 507 k 2023-10-06T13:50:20.388 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:20.388 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:50:20.388 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:20.389 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:50:20.389 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:20.389 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 507 k 2023-10-06T13:50:20.389 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 2.5 M 2023-10-06T13:50:20.389 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:50:20.526 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:50:20.527 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:20.527 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:50:20.527 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:20.527 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:20.527 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:50:20.655 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librados-devel-debuginfo 2023-10-06T13:50:20.777 INFO:teuthology.orchestra.run.smithi083.stdout:rbd-fuse-debuginfo-18.2.0-596.g4af54c4a.el9.x86 1.3 MB/s | 507 kB 00:00 2023-10-06T13:50:20.778 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:20.778 INFO:teuthology.orchestra.run.smithi083.stdout:Total 1.3 MB/s | 507 kB 00:00 2023-10-06T13:50:20.778 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:50:20.781 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:50:20.781 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:50:20.817 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:50:20.818 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:50:20.974 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:50:21.075 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:21.278 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:07:53 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:50:21.421 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:50:21.422 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:50:21.422 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:21.423 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:50:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 108 k 2023-10-06T13:50:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:21.423 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:50:21.423 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:21.423 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:50:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:21.423 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 108 k 2023-10-06T13:50:21.423 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 440 k 2023-10-06T13:50:21.423 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:50:21.485 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:21.636 INFO:teuthology.orchestra.run.smithi019.stdout:librados-devel-debuginfo-18.2.0-596.g4af54c4a.e 508 kB/s | 108 kB 00:00 2023-10-06T13:50:21.637 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:21.637 INFO:teuthology.orchestra.run.smithi019.stdout:Total 505 kB/s | 108 kB 00:00 2023-10-06T13:50:21.637 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:50:21.641 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:50:21.641 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:50:21.676 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:50:21.676 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:50:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:21.775 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:50:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:21.775 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:50:21.797 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:50:21.903 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-debuginfo-2:18.2.0-596.g4af54c4a.el9. 1/1 2023-10-06T13:50:21.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install rbd-mirror-debuginfo 2023-10-06T13:50:22.276 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados-devel-debuginfo-2:18.2.0-596.g4af54c4a.el9. 1/1 2023-10-06T13:50:22.559 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:07:55 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:50:22.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-debuginfo-2:18.2.0-596.g4af54c4a.el9. 1/1 2023-10-06T13:50:22.581 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:22.581 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:50:22.581 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:22.581 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:22.581 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:50:22.696 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:50:22.696 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:50:22.697 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:22.697 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:50:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 51 M 2023-10-06T13:50:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:22.697 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:50:22.697 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:22.697 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:50:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:22.697 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 51 M 2023-10-06T13:50:22.697 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 282 M 2023-10-06T13:50:22.698 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:50:22.751 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install libcephfs2-debuginfo 2023-10-06T13:50:23.373 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:07:55 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:50:23.516 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:50:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:50:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:50:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 4.8 M 2023-10-06T13:50:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:50:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:50:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 4.8 M 2023-10-06T13:50:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 23 M 2023-10-06T13:50:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:50:23.989 INFO:teuthology.orchestra.run.smithi019.stdout:libcephfs2-debuginfo-18.2.0-596.g4af54c4a.el9.x 10 MB/s | 4.8 MB 00:00 2023-10-06T13:50:23.990 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:23.990 INFO:teuthology.orchestra.run.smithi019.stdout:Total 10 MB/s | 4.8 MB 00:00 2023-10-06T13:50:23.991 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:50:23.994 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:50:23.994 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:50:24.061 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:50:24.061 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:50:24.715 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:50:24.807 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_ 1/1 2023-10-06T13:50:25.333 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_ 1/1 2023-10-06T13:50:25.818 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_ 1/1 2023-10-06T13:50:25.818 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:25.818 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:50:25.818 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:25.818 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:25.818 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:50:25.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librados2-debuginfo 2023-10-06T13:50:26.615 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:07:58 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:50:26.713 INFO:teuthology.orchestra.run.smithi083.stdout:rbd-mirror-debuginfo-18.2.0-596.g4af54c4a.el9.x 13 MB/s | 51 MB 00:04 2023-10-06T13:50:26.714 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:26.715 INFO:teuthology.orchestra.run.smithi083.stdout:Total 13 MB/s | 51 MB 00:04 2023-10-06T13:50:26.715 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:50:26.719 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:50:26.719 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:50:26.758 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:50:26.759 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:50:26.759 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:26.759 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:50:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 44 M 2023-10-06T13:50:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:26.759 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:50:26.759 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:26.760 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:50:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:26.760 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 44 M 2023-10-06T13:50:26.760 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 238 M 2023-10-06T13:50:26.760 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:50:27.049 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:50:27.049 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:50:32.754 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:50:32.844 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_ 1/1 2023-10-06T13:50:33.716 INFO:teuthology.orchestra.run.smithi019.stdout:librados2-debuginfo-18.2.0-596.g4af54c4a.el9.x8 6.3 MB/s | 44 MB 00:06 2023-10-06T13:50:33.717 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:33.718 INFO:teuthology.orchestra.run.smithi019.stdout:Total 6.3 MB/s | 44 MB 00:06 2023-10-06T13:50:33.718 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:50:33.722 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:50:33.722 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:50:34.040 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:50:34.040 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:50:34.905 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_ 1/1 2023-10-06T13:50:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_ 1/1 2023-10-06T13:50:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:35.187 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:50:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:35.188 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:35.188 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:50:35.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install rbd-nbd-debuginfo 2023-10-06T13:50:35.977 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:08:08 ago on Fri 06 Oct 2023 01:42:27 PM UTC. 2023-10-06T13:50:36.114 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-10-06T13:50:36.115 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2023-10-06T13:50:36.115 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:36.115 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-10-06T13:50:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 1.2 M 2023-10-06T13:50:36.115 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:36.116 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-10-06T13:50:36.116 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2023-10-06T13:50:36.116 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2023-10-06T13:50:36.116 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:36.116 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 1.2 M 2023-10-06T13:50:36.116 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 5.4 M 2023-10-06T13:50:36.116 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-10-06T13:50:36.416 INFO:teuthology.orchestra.run.smithi083.stdout:rbd-nbd-debuginfo-18.2.0-596.g4af54c4a.el9.x86_ 4.1 MB/s | 1.2 MB 00:00 2023-10-06T13:50:36.417 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:36.417 INFO:teuthology.orchestra.run.smithi083.stdout:Total 4.1 MB/s | 1.2 MB 00:00 2023-10-06T13:50:36.417 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-10-06T13:50:36.420 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-10-06T13:50:36.420 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-10-06T13:50:36.460 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-10-06T13:50:36.460 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-10-06T13:50:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-10-06T13:50:36.782 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:37.227 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-nbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:37.504 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:37.504 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:37.504 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-10-06T13:50:37.504 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:37.504 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:50:37.505 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-10-06T13:50:37.695 DEBUG:teuthology.parallel:result is None 2023-10-06T13:50:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:50:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:50:41.161 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:50:41.450 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_6 1/1 2023-10-06T13:50:41.450 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:50:41.450 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:41.450 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:50:41.614 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librbd1-debuginfo 2023-10-06T13:50:42.231 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:08:14 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:50:42.377 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:50:42.378 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:50:42.378 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:42.378 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:50:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 51 M 2023-10-06T13:50:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:42.378 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:50:42.378 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:42.379 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:50:42.379 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:42.379 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 51 M 2023-10-06T13:50:42.379 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 292 M 2023-10-06T13:50:42.379 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:50:44.794 INFO:teuthology.orchestra.run.smithi019.stdout:librbd1-debuginfo-18.2.0-596.g4af54c4a.el9.x86_ 21 MB/s | 51 MB 00:02 2023-10-06T13:50:44.795 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:44.796 INFO:teuthology.orchestra.run.smithi019.stdout:Total 21 MB/s | 51 MB 00:02 2023-10-06T13:50:44.796 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:50:44.800 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:50:44.800 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:50:45.159 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:50:45.159 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:50:51.450 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:50:51.537 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librbd1-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:53.689 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:53.980 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:50:53.980 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:53.980 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:50:53.980 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:53.980 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:53.980 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:50:54.149 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-cephfs-debuginfo 2023-10-06T13:50:54.768 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:08:26 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:50:54.912 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:50:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:54.913 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:50:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:50:54.913 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 593 k 2023-10-06T13:50:54.913 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:50:54.914 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:54.914 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:50:54.914 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:54.914 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 593 k 2023-10-06T13:50:54.914 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 2.0 M 2023-10-06T13:50:54.914 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:50:55.330 INFO:teuthology.orchestra.run.smithi019.stdout:python3-cephfs-debuginfo-18.2.0-596.g4af54c4a.e 1.4 MB/s | 593 kB 00:00 2023-10-06T13:50:55.331 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:55.331 INFO:teuthology.orchestra.run.smithi019.stdout:Total 1.4 MB/s | 593 kB 00:00 2023-10-06T13:50:55.331 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:50:55.335 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:50:55.335 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:50:55.375 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:50:55.375 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:50:55.541 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:50:55.647 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-debuginfo-2:18.2.0-596.g4af54c4a.el9. 1/1 2023-10-06T13:50:56.057 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: python3-cephfs-debuginfo-2:18.2.0-596.g4af54c4a.el9. 1/1 2023-10-06T13:50:56.331 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-debuginfo-2:18.2.0-596.g4af54c4a.el9. 1/1 2023-10-06T13:50:56.331 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:56.331 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:50:56.331 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:56.331 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:56.331 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:50:56.485 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rados-debuginfo 2023-10-06T13:50:57.107 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:08:29 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:50:57.250 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:50:57.251 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:57.251 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:50:57.251 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:57.251 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:50:57.252 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 770 k 2023-10-06T13:50:57.252 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:57.252 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:50:57.252 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:57.252 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:50:57.252 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:57.252 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 770 k 2023-10-06T13:50:57.252 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 2.7 M 2023-10-06T13:50:57.252 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:50:57.542 INFO:teuthology.orchestra.run.smithi019.stdout:python3-rados-debuginfo-18.2.0-596.g4af54c4a.el 2.6 MB/s | 770 kB 00:00 2023-10-06T13:50:57.542 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:57.542 INFO:teuthology.orchestra.run.smithi019.stdout:Total 2.6 MB/s | 770 kB 00:00 2023-10-06T13:50:57.543 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:50:57.546 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:50:57.546 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:50:57.585 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:50:57.586 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:50:57.776 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:50:57.884 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-debuginfo-2:18.2.0-596.g4af54c4a.el9.x 1/1 2023-10-06T13:50:58.281 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: python3-rados-debuginfo-2:18.2.0-596.g4af54c4a.el9.x 1/1 2023-10-06T13:50:58.563 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-debuginfo-2:18.2.0-596.g4af54c4a.el9.x 1/1 2023-10-06T13:50:58.563 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:58.563 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:50:58.564 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:50:58.564 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:58.564 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:50:58.730 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rbd-debuginfo 2023-10-06T13:50:59.354 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:08:31 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:50:59.498 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:50:59.499 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:59.499 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:50:59.499 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:59.499 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:50:59.499 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 1.1 M 2023-10-06T13:50:59.500 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:59.500 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:50:59.500 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:50:59.500 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:50:59.500 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:50:59.500 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 1.1 M 2023-10-06T13:50:59.500 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 4.3 M 2023-10-06T13:50:59.500 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:50:59.936 INFO:teuthology.orchestra.run.smithi019.stdout:python3-rbd-debuginfo-18.2.0-596.g4af54c4a.el9. 2.6 MB/s | 1.1 MB 00:00 2023-10-06T13:50:59.937 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:50:59.937 INFO:teuthology.orchestra.run.smithi019.stdout:Total 2.6 MB/s | 1.1 MB 00:00 2023-10-06T13:50:59.937 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:50:59.940 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:50:59.940 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:50:59.983 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:50:59.983 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:51:00.207 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:51:00.312 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:51:00.722 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: python3-rbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:51:01.023 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:51:01.023 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:01.024 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:51:01.024 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:51:01.024 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:01.024 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:51:01.194 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rgw-debuginfo 2023-10-06T13:51:01.816 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:08:33 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:51:01.959 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:51:01.960 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:51:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:51:01.961 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:51:01.961 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:51:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 337 k 2023-10-06T13:51:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:01.961 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:51:01.961 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:51:01.961 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:51:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:01.962 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 337 k 2023-10-06T13:51:01.962 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 1.0 M 2023-10-06T13:51:01.962 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:51:02.214 INFO:teuthology.orchestra.run.smithi019.stdout:python3-rgw-debuginfo-18.2.0-596.g4af54c4a.el9. 1.3 MB/s | 337 kB 00:00 2023-10-06T13:51:02.215 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:51:02.215 INFO:teuthology.orchestra.run.smithi019.stdout:Total 1.3 MB/s | 337 kB 00:00 2023-10-06T13:51:02.215 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:51:02.218 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:51:02.218 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:51:02.256 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:51:02.256 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:51:02.398 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:51:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:51:02.908 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: python3-rgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:51:03.188 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86 1/1 2023-10-06T13:51:03.188 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:03.188 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:51:03.188 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:51:03.188 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:03.188 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:51:03.387 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-fuse-debuginfo 2023-10-06T13:51:04.010 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:08:36 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:51:04.153 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:51:04.154 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:51:04.154 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:51:04.154 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:51:04.154 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:51:04.154 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 507 k 2023-10-06T13:51:04.154 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:04.154 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:51:04.154 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:51:04.154 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:51:04.155 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:04.155 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 507 k 2023-10-06T13:51:04.155 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 2.5 M 2023-10-06T13:51:04.155 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:51:04.534 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-fuse-debuginfo-18.2.0-596.g4af54c4a.el9.x86 1.3 MB/s | 507 kB 00:00 2023-10-06T13:51:04.535 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:51:04.535 INFO:teuthology.orchestra.run.smithi019.stdout:Total 1.3 MB/s | 507 kB 00:00 2023-10-06T13:51:04.535 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:51:04.538 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:51:04.539 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:51:04.577 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:51:04.577 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:51:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:51:04.844 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:51:05.210 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:51:05.480 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:51:05.481 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:05.481 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:51:05.481 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:51:05.481 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:05.481 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:51:05.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-mirror-debuginfo 2023-10-06T13:51:06.273 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:08:38 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:51:06.416 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:51:06.417 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:51:06.417 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:51:06.417 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:51:06.417 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:51:06.417 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 51 M 2023-10-06T13:51:06.417 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:06.417 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:51:06.418 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:51:06.418 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:51:06.418 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:06.418 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 51 M 2023-10-06T13:51:06.418 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 282 M 2023-10-06T13:51:06.418 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:51:09.170 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-mirror-debuginfo-18.2.0-596.g4af54c4a.el9.x 18 MB/s | 51 MB 00:02 2023-10-06T13:51:09.171 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:51:09.171 INFO:teuthology.orchestra.run.smithi019.stdout:Total 18 MB/s | 51 MB 00:02 2023-10-06T13:51:09.172 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:51:09.176 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:51:09.176 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:51:09.525 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:51:09.525 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:51:15.605 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:51:15.703 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_ 1/1 2023-10-06T13:51:17.693 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_ 1/1 2023-10-06T13:51:17.986 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_ 1/1 2023-10-06T13:51:17.986 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:17.986 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:51:17.986 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:51:17.986 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:17.987 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:51:18.174 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-nbd-debuginfo 2023-10-06T13:51:18.790 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:08:50 ago on Fri 06 Oct 2023 01:42:28 PM UTC. 2023-10-06T13:51:18.935 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-10-06T13:51:18.936 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:51:18.936 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-10-06T13:51:18.936 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:51:18.936 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-10-06T13:51:18.936 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-debuginfo x86_64 2:18.2.0-596.g4af54c4a.el9 ceph 1.2 M 2023-10-06T13:51:18.937 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:18.937 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-10-06T13:51:18.937 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-10-06T13:51:18.937 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-10-06T13:51:18.937 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:18.937 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 1.2 M 2023-10-06T13:51:18.937 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 5.4 M 2023-10-06T13:51:18.937 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-10-06T13:51:19.239 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-nbd-debuginfo-18.2.0-596.g4af54c4a.el9.x86_ 4.1 MB/s | 1.2 MB 00:00 2023-10-06T13:51:19.240 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-10-06T13:51:19.240 INFO:teuthology.orchestra.run.smithi019.stdout:Total 4.1 MB/s | 1.2 MB 00:00 2023-10-06T13:51:19.240 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-10-06T13:51:19.243 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-10-06T13:51:19.244 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-10-06T13:51:19.286 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-10-06T13:51:19.287 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-10-06T13:51:19.532 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-10-06T13:51:19.631 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:51:20.049 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-nbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:51:20.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 1/1 2023-10-06T13:51:20.337 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:20.337 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-10-06T13:51:20.337 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-debuginfo-2:18.2.0-596.g4af54c4a.el9.x86_64 2023-10-06T13:51:20.337 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:20.337 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-10-06T13:51:20.506 DEBUG:teuthology.parallel:result is None 2023-10-06T13:51:20.506 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4af54c4a1461e3792e9783e337bb62ce429214f3 2023-10-06T13:51:20.649 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T13:51:20.670 INFO:teuthology.orchestra.run.smithi019.stdout:18.2.0-596.g4af54c4a.el9 2023-10-06T13:51:20.670 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-596.g4af54c4a.el9 2023-10-06T13:51:20.671 INFO:teuthology.task.install:The correct ceph version 18.2.0-596.g4af54c4a is installed. 2023-10-06T13:51:20.671 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4af54c4a1461e3792e9783e337bb62ce429214f3 2023-10-06T13:51:20.808 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T13:51:20.833 INFO:teuthology.orchestra.run.smithi083.stdout:18.2.0-596.g4af54c4a.el9 2023-10-06T13:51:20.833 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-596.g4af54c4a.el9 2023-10-06T13:51:20.833 INFO:teuthology.task.install:The correct ceph version 18.2.0-596.g4af54c4a is installed. 2023-10-06T13:51:20.834 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T13:51:20.834 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:20.834 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T13:51:20.867 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:51:20.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T13:51:20.906 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T13:51:20.906 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:20.906 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T13:51:20.937 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T13:51:21.008 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:51:21.008 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T13:51:21.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T13:51:21.107 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T13:51:21.108 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:21.108 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T13:51:21.137 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T13:51:21.208 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:51:21.208 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T13:51:21.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T13:51:21.307 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T13:51:21.307 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:21.307 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T13:51:21.337 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T13:51:21.408 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:51:21.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T13:51:21.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T13:51:21.504 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T13:51:21.574 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T13:51:21.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /var/log/ceph 2023-10-06T13:51:21.577 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /var/log/ceph 2023-10-06T13:51:21.605 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T13:51:21.605 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T13:51:21.649 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T13:51:21.676 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T13:51:21.676 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T13:51:21.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T13:51:21.756 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T13:51:21.756 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 120, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T13:51:21.756 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609', 'branch': 'wip-yuri7-testing-2023-10-04-1350-reef', 'description': 'rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412609', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri7-testing-2023-10-04-1350-reef'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 120, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request'], 'sha1': '4af54c4a1461e3792e9783e337bb62ce429214f3', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': '4af54c4a1461e3792e9783e337bb62ce429214f3'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri7-testing-2023-10-04-1350-reef', 'sha1': '4af54c4a1461e3792e9783e337bb62ce429214f3'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 2590, 'sha1': '4af54c4a1461e3792e9783e337bb62ce429214f3', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri7-testing-2023-10-04-1350-reef', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '4af54c4a1461e3792e9783e337bb62ce429214f3', 'targets': {'smithi019.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKFVHDP42vImDPuf04uS5rW92D9ajR1cl+iGPKEkXHSvOkbk7aW87koA584qj1YGCwIYmkWLzgrQm43lX51y5DE=', 'smithi083.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPAnIPUznwUocjlTpOrgy72Ol5ISFYjF2B2TswcYhFAoqtpeWOZpkDxB8ufWw295eDlKNnYynEE4qzouoO9i7dk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'workunit': {'clients': {'client.0': ['rados/test.sh']}, 'env': {'ALLOW_TIMEOUTS': '1'}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:58:21', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T13:51:21.757 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T13:51:21.788 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T13:51:21.803 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T13:51:21.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T13:51:21.871 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:21.871 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T13:51:21.926 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T13:51:21.926 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2023-10-06T13:51:21.980 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T13:51:21.981 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T13:51:21.981 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 662 Links: 1 2023-10-06T13:51:21.981 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T13:51:21.981 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T13:51:21.981 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2023-10-06 13:44:10.889511521 +0000 2023-10-06T13:51:21.981 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2023-10-06 13:41:43.260536647 +0000 2023-10-06T13:51:21.981 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2023-10-06 13:41:43.260536647 +0000 2023-10-06T13:51:21.982 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2023-10-06 13:41:43.260536647 +0000 2023-10-06T13:51:21.982 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T13:51:22.049 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2023-10-06T13:51:22.050 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2023-10-06T13:51:22.050 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000190906 s, 2.7 MB/s 2023-10-06T13:51:22.051 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T13:51:22.106 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2023-10-06T13:51:22.162 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T13:51:22.162 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T13:51:22.162 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 682 Links: 1 2023-10-06T13:51:22.162 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T13:51:22.162 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T13:51:22.162 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2023-10-06 13:44:10.889511521 +0000 2023-10-06T13:51:22.162 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2023-10-06 13:41:43.758532449 +0000 2023-10-06T13:51:22.162 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2023-10-06 13:41:43.758532449 +0000 2023-10-06T13:51:22.162 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2023-10-06 13:41:43.758532449 +0000 2023-10-06T13:51:22.163 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T13:51:22.230 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2023-10-06T13:51:22.231 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2023-10-06T13:51:22.231 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000173673 s, 2.9 MB/s 2023-10-06T13:51:22.232 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T13:51:22.288 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2023-10-06T13:51:22.342 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T13:51:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T13:51:22.343 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 702 Links: 1 2023-10-06T13:51:22.343 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T13:51:22.343 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T13:51:22.343 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2023-10-06 13:44:10.889511521 +0000 2023-10-06T13:51:22.343 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2023-10-06 13:41:44.256528251 +0000 2023-10-06T13:51:22.343 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2023-10-06 13:41:44.257528242 +0000 2023-10-06T13:51:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2023-10-06 13:41:44.256528251 +0000 2023-10-06T13:51:22.343 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T13:51:22.411 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2023-10-06T13:51:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2023-10-06T13:51:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000175493 s, 2.9 MB/s 2023-10-06T13:51:22.413 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T13:51:22.469 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2023-10-06T13:51:22.525 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T13:51:22.525 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T13:51:22.525 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 720 Links: 1 2023-10-06T13:51:22.525 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T13:51:22.525 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T13:51:22.526 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2023-10-06 13:44:10.890511512 +0000 2023-10-06T13:51:22.526 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2023-10-06 13:41:44.758524019 +0000 2023-10-06T13:51:22.526 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2023-10-06 13:41:44.758524019 +0000 2023-10-06T13:51:22.526 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2023-10-06 13:41:44.758524019 +0000 2023-10-06T13:51:22.526 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T13:51:22.595 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2023-10-06T13:51:22.595 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2023-10-06T13:51:22.595 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000193286 s, 2.6 MB/s 2023-10-06T13:51:22.596 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T13:51:22.652 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T13:51:22.653 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:51:22.653 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T13:51:22.671 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T13:51:22.671 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2023-10-06T13:51:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T13:51:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T13:51:22.728 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 662 Links: 1 2023-10-06T13:51:22.728 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T13:51:22.728 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T13:51:22.728 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2023-10-06 13:44:07.915928353 +0000 2023-10-06T13:51:22.728 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2023-10-06 13:41:32.929015718 +0000 2023-10-06T13:51:22.728 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2023-10-06 13:41:32.929015718 +0000 2023-10-06T13:51:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2023-10-06 13:41:32.929015718 +0000 2023-10-06T13:51:22.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T13:51:22.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-10-06T13:51:22.797 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-10-06T13:51:22.797 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000216513 s, 2.4 MB/s 2023-10-06T13:51:22.798 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T13:51:22.855 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2023-10-06T13:51:22.914 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T13:51:22.914 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T13:51:22.914 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 684 Links: 1 2023-10-06T13:51:22.914 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T13:51:22.914 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T13:51:22.915 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2023-10-06 13:44:07.915928353 +0000 2023-10-06T13:51:22.915 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2023-10-06 13:41:33.434003557 +0000 2023-10-06T13:51:22.915 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2023-10-06 13:41:33.434003557 +0000 2023-10-06T13:51:22.915 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2023-10-06 13:41:33.434003557 +0000 2023-10-06T13:51:22.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T13:51:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-10-06T13:51:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-10-06T13:51:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000202785 s, 2.5 MB/s 2023-10-06T13:51:22.981 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T13:51:23.037 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2023-10-06T13:51:23.092 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T13:51:23.092 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T13:51:23.092 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 703 Links: 1 2023-10-06T13:51:23.092 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T13:51:23.092 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T13:51:23.092 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2023-10-06 13:44:07.916928330 +0000 2023-10-06T13:51:23.092 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2023-10-06 13:41:33.913991998 +0000 2023-10-06T13:51:23.093 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2023-10-06 13:41:33.913991998 +0000 2023-10-06T13:51:23.093 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2023-10-06 13:41:33.913991998 +0000 2023-10-06T13:51:23.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T13:51:23.158 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-10-06T13:51:23.158 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-10-06T13:51:23.159 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000236543 s, 2.2 MB/s 2023-10-06T13:51:23.160 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T13:51:23.216 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2023-10-06T13:51:23.271 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T13:51:23.271 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T13:51:23.271 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 720 Links: 1 2023-10-06T13:51:23.271 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T13:51:23.271 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T13:51:23.271 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2023-10-06 13:44:07.916928330 +0000 2023-10-06T13:51:23.271 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2023-10-06 13:41:34.409980054 +0000 2023-10-06T13:51:23.271 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2023-10-06 13:41:34.409980054 +0000 2023-10-06T13:51:23.272 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2023-10-06 13:41:34.409980054 +0000 2023-10-06T13:51:23.272 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T13:51:23.336 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-10-06T13:51:23.336 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-10-06T13:51:23.336 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000198282 s, 2.6 MB/s 2023-10-06T13:51:23.337 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T13:51:23.392 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T13:51:23.393 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T13:51:23.393 INFO:tasks.ceph:Generating config... 2023-10-06T13:51:23.394 INFO:tasks.ceph:[client] debug monc = 20 2023-10-06T13:51:23.394 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T13:51:23.394 INFO:tasks.ceph:[client] debug objecter = 20 2023-10-06T13:51:23.394 INFO:tasks.ceph:[client] debug rados = 20 2023-10-06T13:51:23.394 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-10-06T13:51:23.394 INFO:tasks.ceph:[global] mon election default strategy = 1 2023-10-06T13:51:23.394 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2023-10-06T13:51:23.395 INFO:tasks.ceph:[global] ms type = async 2023-10-06T13:51:23.395 INFO:tasks.ceph:[global] osd heartbeat grace = 80 2023-10-06T13:51:23.395 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T13:51:23.395 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T13:51:23.395 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T13:51:23.395 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T13:51:23.395 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T13:51:23.395 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-10-06T13:51:23.395 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2023-10-06T13:51:23.395 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-10-06T13:51:23.395 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-10-06T13:51:23.395 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-10-06T13:51:23.395 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-10-06T13:51:23.396 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2023-10-06T13:51:23.396 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-10-06T13:51:23.396 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-06T13:51:23.396 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-06T13:51:23.396 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-06T13:51:23.396 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T13:51:23.396 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T13:51:23.396 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-10-06T13:51:23.396 INFO:tasks.ceph:[osd] debug bluefs = 1 2023-10-06T13:51:23.396 INFO:tasks.ceph:[osd] debug bluestore = 1 2023-10-06T13:51:23.396 INFO:tasks.ceph:[osd] debug monc = 20 2023-10-06T13:51:23.396 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T13:51:23.397 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T13:51:23.397 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-06T13:51:23.397 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T13:51:23.397 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T13:51:23.397 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T13:51:23.397 INFO:tasks.ceph:[osd] osd class default list = * 2023-10-06T13:51:23.397 INFO:tasks.ceph:[osd] osd class load list = * 2023-10-06T13:51:23.397 INFO:tasks.ceph:[osd] osd client watch timeout = 120 2023-10-06T13:51:23.397 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-10-06T13:51:23.397 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-10-06T13:51:23.397 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T13:51:23.397 INFO:tasks.ceph:[osd] osd fast shutdown = False 2023-10-06T13:51:23.397 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-10-06T13:51:23.398 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2023-10-06T13:51:23.398 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T13:51:23.398 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-10-06T13:51:23.398 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-10-06T13:51:23.398 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-10-06T13:51:23.398 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T13:51:23.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T13:51:23.453 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T13:51:23.456 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T13:51:23.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T13:51:23.535 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.19'), ('mon.c', '[v2:172.21.15.19:3301,v1:172.21.15.19:6790]'), ('mon.b', '172.21.15.83')] 2023-10-06T13:51:23.535 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.19,[v2:172.21.15.19:3301,v1:172.21.15.19:6790],172.21.15.83', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 120, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-10-06T13:51:23.536 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:23.536 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T13:51:23.591 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.19 --addv c '[v2:172.21.15.19:3301,v1:172.21.15.19:6790]' --add b 172.21.15.83 --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T13:51:23.676 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T13:51:23.676 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: generated fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:51:23.676 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = pacific 2023-10-06T13:51:23.676 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 0 2023-10-06T13:51:23.676 INFO:teuthology.orchestra.run.smithi019.stdout:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:51:23.676 INFO:teuthology.orchestra.run.smithi019.stdout:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:51:23.676 INFO:teuthology.orchestra.run.smithi019.stdout:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:51:23.676 INFO:teuthology.orchestra.run.smithi019.stdout:min_mon_release 16 (pacific) 2023-10-06T13:51:23.676 INFO:teuthology.orchestra.run.smithi019.stdout:election_strategy: 1 2023-10-06T13:51:23.677 INFO:teuthology.orchestra.run.smithi019.stdout:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:51:23.677 INFO:teuthology.orchestra.run.smithi019.stdout:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:51:23.677 INFO:teuthology.orchestra.run.smithi019.stdout:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:51:23.677 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T13:51:23.678 DEBUG:teuthology.orchestra.run.smithi019:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T13:51:23.692 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c5a90f7c-81ec-43a3-ab47-af6ce5729ec5... 2023-10-06T13:51:23.693 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T13:51:23.735 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T13:51:23.786 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2023-10-06T13:51:23.786 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2023-10-06T13:51:23.787 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T13:51:23.787 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2023-10-06T13:51:23.787 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.787 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2023-10-06T13:51:23.787 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.787 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2023-10-06T13:51:23.787 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.787 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2023-10-06T13:51:23.788 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2023-10-06T13:51:23.788 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.788 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2023-10-06T13:51:23.788 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2023-10-06T13:51:23.788 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.788 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T13:51:23.788 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2023-10-06T13:51:23.788 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.788 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T13:51:23.788 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T13:51:23.789 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.789 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2023-10-06T13:51:23.789 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.789 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2023-10-06T13:51:23.789 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2023-10-06T13:51:23.789 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2023-10-06T13:51:23.789 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2023-10-06T13:51:23.789 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2023-10-06T13:51:23.789 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2023-10-06T13:51:23.789 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2023-10-06T13:51:23.789 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T13:51:23.790 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T13:51:23.790 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2023-10-06T13:51:23.790 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.790 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T13:51:23.790 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.790 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2023-10-06T13:51:23.790 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.790 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2023-10-06T13:51:23.790 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.790 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2023-10-06T13:51:23.790 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2023-10-06T13:51:23.790 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2023-10-06T13:51:23.791 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.19,[v2:172.21.15.19:3301,v1:172.21.15.19:6790],172.21.15.83" 2023-10-06T13:51:23.791 INFO:teuthology.orchestra.run.smithi083.stdout: mon client directed command retry = 5 2023-10-06T13:51:23.791 INFO:teuthology.orchestra.run.smithi083.stdout: mon election default strategy = 1 2023-10-06T13:51:23.791 INFO:teuthology.orchestra.run.smithi083.stdout: ms inject socket failures = 5000 2023-10-06T13:51:23.791 INFO:teuthology.orchestra.run.smithi083.stdout: ms type = async 2023-10-06T13:51:23.791 INFO:teuthology.orchestra.run.smithi083.stdout: osd heartbeat grace = 80 2023-10-06T13:51:23.791 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:51:23.791 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.791 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2023-10-06T13:51:23.791 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2023-10-06T13:51:23.791 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.791 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2023-10-06T13:51:23.792 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2023-10-06T13:51:23.792 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2023-10-06T13:51:23.792 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.792 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2023-10-06T13:51:23.792 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2023-10-06T13:51:23.792 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.792 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2023-10-06T13:51:23.792 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2023-10-06T13:51:23.792 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2023-10-06T13:51:23.792 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.792 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2023-10-06T13:51:23.792 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2023-10-06T13:51:23.793 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.793 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2023-10-06T13:51:23.793 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.793 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2023-10-06T13:51:23.793 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.793 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2023-10-06T13:51:23.793 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.793 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2023-10-06T13:51:23.793 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2023-10-06T13:51:23.793 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.793 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2023-10-06T13:51:23.793 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2023-10-06T13:51:23.794 INFO:teuthology.orchestra.run.smithi083.stdout: bdev async discard = True 2023-10-06T13:51:23.794 INFO:teuthology.orchestra.run.smithi083.stdout: bdev enable discard = True 2023-10-06T13:51:23.794 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore allocator = bitmap 2023-10-06T13:51:23.794 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2023-10-06T13:51:23.794 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2023-10-06T13:51:23.794 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore zero block detection = True 2023-10-06T13:51:23.794 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 1 2023-10-06T13:51:23.794 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 1 2023-10-06T13:51:23.794 INFO:teuthology.orchestra.run.smithi083.stdout: debug monc = 20 2023-10-06T13:51:23.794 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2023-10-06T13:51:23.794 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2023-10-06T13:51:23.794 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 10 2023-10-06T13:51:23.795 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T13:51:23.795 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2023-10-06T13:51:23.795 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2023-10-06T13:51:23.795 INFO:teuthology.orchestra.run.smithi083.stdout: osd class default list = * 2023-10-06T13:51:23.795 INFO:teuthology.orchestra.run.smithi083.stdout: osd class load list = * 2023-10-06T13:51:23.795 INFO:teuthology.orchestra.run.smithi083.stdout: osd client watch timeout = 120 2023-10-06T13:51:23.795 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify cached snaps = True 2023-10-06T13:51:23.795 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify missing on start = True 2023-10-06T13:51:23.795 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2023-10-06T13:51:23.795 INFO:teuthology.orchestra.run.smithi083.stdout: osd fast shutdown = False 2023-10-06T13:51:23.795 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock override recovery settings = True 2023-10-06T13:51:23.795 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2023-10-06T13:51:23.795 INFO:teuthology.orchestra.run.smithi083.stdout: osd op queue = debug_random 2023-10-06T13:51:23.796 INFO:teuthology.orchestra.run.smithi083.stdout: osd op queue cut off = debug_random 2023-10-06T13:51:23.796 INFO:teuthology.orchestra.run.smithi083.stdout: osd shutdown pgref assert = True 2023-10-06T13:51:23.796 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.796 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2023-10-06T13:51:23.796 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2023-10-06T13:51:23.796 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2023-10-06T13:51:23.796 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2023-10-06T13:51:23.796 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2023-10-06T13:51:23.797 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2023-10-06T13:51:23.798 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2023-10-06T13:51:23.798 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2023-10-06T13:51:23.798 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.798 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2023-10-06T13:51:23.798 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2023-10-06T13:51:23.798 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2023-10-06T13:51:23.798 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2023-10-06T13:51:23.798 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2023-10-06T13:51:23.798 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2023-10-06T13:51:23.799 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2023-10-06T13:51:23.799 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2023-10-06T13:51:23.799 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2023-10-06T13:51:23.799 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2023-10-06T13:51:23.799 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2023-10-06T13:51:23.799 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.799 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T13:51:23.799 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T13:51:23.799 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T13:51:23.799 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.800 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T13:51:23.800 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T13:51:23.800 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T13:51:23.800 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.800 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2023-10-06T13:51:23.800 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2023-10-06T13:51:23.800 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.800 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2023-10-06T13:51:23.800 INFO:teuthology.orchestra.run.smithi083.stdout: mon min osdmap epochs = 50 2023-10-06T13:51:23.800 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd crush smoke test = False 2023-10-06T13:51:23.800 INFO:teuthology.orchestra.run.smithi083.stdout: mon osdmap full prune interval = 2 2023-10-06T13:51:23.800 INFO:teuthology.orchestra.run.smithi083.stdout: mon osdmap full prune min = 15 2023-10-06T13:51:23.801 INFO:teuthology.orchestra.run.smithi083.stdout: mon osdmap full prune txsize = 2 2023-10-06T13:51:23.801 INFO:teuthology.orchestra.run.smithi083.stdout: mon scrub interval = 300 2023-10-06T13:51:23.801 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on pool no app = False 2023-10-06T13:51:23.801 INFO:teuthology.orchestra.run.smithi083.stdout: paxos service trim min = 10 2023-10-06T13:51:23.801 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-10-06T13:51:23.801 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2023-10-06T13:51:23.801 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2023-10-06T13:51:23.801 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2023-10-06T13:51:23.801 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2023-10-06T13:51:23.801 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T13:51:23.801 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T13:51:23.801 INFO:teuthology.orchestra.run.smithi083.stdout: debug monc = 20 2023-10-06T13:51:23.802 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2023-10-06T13:51:23.802 INFO:teuthology.orchestra.run.smithi083.stdout: debug objecter = 20 2023-10-06T13:51:23.802 INFO:teuthology.orchestra.run.smithi083.stdout: debug rados = 20 2023-10-06T13:51:23.802 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2023-10-06T13:51:23.802 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2023-10-06T13:51:23.802 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2023-10-06T13:51:23.804 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2023-10-06T13:51:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2023-10-06T13:51:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T13:51:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2023-10-06T13:51:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2023-10-06T13:51:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2023-10-06T13:51:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2023-10-06T13:51:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2023-10-06T13:51:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2023-10-06T13:51:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2023-10-06T13:51:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T13:51:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2023-10-06T13:51:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T13:51:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T13:51:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2023-10-06T13:51:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2023-10-06T13:51:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2023-10-06T13:51:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2023-10-06T13:51:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2023-10-06T13:51:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2023-10-06T13:51:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2023-10-06T13:51:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2023-10-06T13:51:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T13:51:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T13:51:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: mon_allow_pool_size_one = true 2023-10-06T13:51:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T13:51:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2023-10-06T13:51:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2023-10-06T13:51:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2023-10-06T13:51:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2023-10-06T13:51:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: mon health detail to clog = false 2023-10-06T13:51:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = "172.21.15.19,[v2:172.21.15.19:3301,v1:172.21.15.19:6790],172.21.15.83" 2023-10-06T13:51:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: mon client directed command retry = 5 2023-10-06T13:51:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: mon election default strategy = 1 2023-10-06T13:51:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: ms inject socket failures = 5000 2023-10-06T13:51:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: ms type = async 2023-10-06T13:51:23.812 INFO:teuthology.orchestra.run.smithi019.stdout: osd heartbeat grace = 80 2023-10-06T13:51:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:51:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.813 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2023-10-06T13:51:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2023-10-06T13:51:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2023-10-06T13:51:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2023-10-06T13:51:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock profile = high_recovery_ops 2023-10-06T13:51:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2023-10-06T13:51:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2023-10-06T13:51:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2023-10-06T13:51:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2023-10-06T13:51:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2023-10-06T13:51:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2023-10-06T13:51:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2023-10-06T13:51:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2023-10-06T13:51:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2023-10-06T13:51:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2023-10-06T13:51:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2023-10-06T13:51:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2023-10-06T13:51:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2023-10-06T13:51:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2023-10-06T13:51:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: bdev async discard = True 2023-10-06T13:51:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: bdev enable discard = True 2023-10-06T13:51:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore allocator = bitmap 2023-10-06T13:51:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore block size = 96636764160 2023-10-06T13:51:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore fsck on mount = True 2023-10-06T13:51:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore zero block detection = True 2023-10-06T13:51:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluefs = 1 2023-10-06T13:51:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluestore = 1 2023-10-06T13:51:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: debug monc = 20 2023-10-06T13:51:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2023-10-06T13:51:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 20 2023-10-06T13:51:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: debug rocksdb = 10 2023-10-06T13:51:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T13:51:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd full ratio = 0.9 2023-10-06T13:51:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd nearfull ratio = 0.8 2023-10-06T13:51:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: osd class default list = * 2023-10-06T13:51:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: osd class load list = * 2023-10-06T13:51:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: osd client watch timeout = 120 2023-10-06T13:51:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify cached snaps = True 2023-10-06T13:51:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify missing on start = True 2023-10-06T13:51:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: osd failsafe full ratio = 0.95 2023-10-06T13:51:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: osd fast shutdown = False 2023-10-06T13:51:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock override recovery settings = True 2023-10-06T13:51:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: osd objectstore = bluestore 2023-10-06T13:51:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: osd op queue = debug_random 2023-10-06T13:51:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: osd op queue cut off = debug_random 2023-10-06T13:51:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: osd shutdown pgref assert = True 2023-10-06T13:51:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.818 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2023-10-06T13:51:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2023-10-06T13:51:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2023-10-06T13:51:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2023-10-06T13:51:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2023-10-06T13:51:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2023-10-06T13:51:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2023-10-06T13:51:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: mgr/telemetry/nag = false 2023-10-06T13:51:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.818 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2023-10-06T13:51:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2023-10-06T13:51:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2023-10-06T13:51:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2023-10-06T13:51:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2023-10-06T13:51:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2023-10-06T13:51:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2023-10-06T13:51:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2023-10-06T13:51:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2023-10-06T13:51:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2023-10-06T13:51:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2023-10-06T13:51:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T13:51:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T13:51:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T13:51:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T13:51:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T13:51:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T13:51:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: # 1m isn't quite enough 2023-10-06T13:51:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: mon_down_mkfs_grace = 2m 2023-10-06T13:51:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_filestore_osds = false 2023-10-06T13:51:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: mon min osdmap epochs = 50 2023-10-06T13:51:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd crush smoke test = False 2023-10-06T13:51:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: mon osdmap full prune interval = 2 2023-10-06T13:51:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: mon osdmap full prune min = 15 2023-10-06T13:51:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: mon osdmap full prune txsize = 2 2023-10-06T13:51:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: mon scrub interval = 300 2023-10-06T13:51:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on pool no app = False 2023-10-06T13:51:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: paxos service trim min = 10 2023-10-06T13:51:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:51:23.821 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2023-10-06T13:51:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2023-10-06T13:51:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2023-10-06T13:51:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2023-10-06T13:51:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T13:51:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T13:51:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: debug monc = 20 2023-10-06T13:51:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2023-10-06T13:51:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: debug objecter = 20 2023-10-06T13:51:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: debug rados = 20 2023-10-06T13:51:23.822 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2023-10-06T13:51:23.822 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.c] 2023-10-06T13:51:23.822 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.b] 2023-10-06T13:51:23.824 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T13:51:23.824 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T13:51:23.923 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T13:51:23.923 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:23.923 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T13:51:23.938 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:23.938 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T13:51:23.993 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi019.front.sepia.ceph.com 2023-10-06T13:51:23.993 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:23.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T13:51:23.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T13:51:24.079 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:24.079 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T13:51:24.133 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi083.front.sepia.ceph.com 2023-10-06T13:51:24.133 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:51:24.134 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T13:51:24.134 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T13:51:24.173 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:51:24.173 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T13:51:24.228 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T13:51:24.228 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T13:51:24.228 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T13:51:24.295 INFO:teuthology.orchestra.run.smithi019.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T13:51:24.295 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T13:51:24.358 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T13:51:24.360 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T13:51:24.360 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T13:51:24.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T13:51:24.408 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T13:51:24.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T13:51:24.548 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T13:51:24.563 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T13:51:24.563 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T13:51:24.563 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T13:51:24.594 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T13:51:24.594 INFO:tasks.ceph:role: osd.0 2023-10-06T13:51:24.594 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi019.front.sepia.ceph.com 2023-10-06T13:51:24.594 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T13:51:24.672 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T13:51:24.672 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T13:51:24.673 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T13:51:24.673 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-06T13:51:24.673 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T13:51:24.673 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2023-10-06T13:51:24.673 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T13:51:24.673 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-06T13:51:24.673 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T13:51:24.674 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T13:51:25.152 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2023-10-06T13:51:25.154 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2023-10-06T13:51:25.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T13:51:25.283 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T13:51:25.321 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T13:51:25.392 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T13:51:25.393 INFO:tasks.ceph:role: osd.1 2023-10-06T13:51:25.393 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi019.front.sepia.ceph.com 2023-10-06T13:51:25.393 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T13:51:25.471 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T13:51:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T13:51:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T13:51:25.472 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-06T13:51:25.473 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T13:51:25.473 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2023-10-06T13:51:25.473 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T13:51:25.473 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-06T13:51:25.474 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T13:51:25.474 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T13:51:25.933 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2023-10-06T13:51:25.935 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2023-10-06T13:51:25.935 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T13:51:26.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T13:51:26.063 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T13:51:26.134 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T13:51:26.135 INFO:tasks.ceph:role: osd.2 2023-10-06T13:51:26.135 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi019.front.sepia.ceph.com 2023-10-06T13:51:26.135 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T13:51:26.212 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T13:51:26.212 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T13:51:26.212 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T13:51:26.212 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-06T13:51:26.212 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T13:51:26.213 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2023-10-06T13:51:26.213 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T13:51:26.213 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-06T13:51:26.213 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T13:51:26.213 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T13:51:26.740 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2023-10-06T13:51:26.741 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2023-10-06T13:51:26.742 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T13:51:26.859 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T13:51:26.895 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T13:51:26.965 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T13:51:26.965 INFO:tasks.ceph:role: osd.3 2023-10-06T13:51:26.965 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi019.front.sepia.ceph.com 2023-10-06T13:51:26.966 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T13:51:27.043 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T13:51:27.044 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T13:51:27.044 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T13:51:27.044 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-06T13:51:27.044 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T13:51:27.044 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2023-10-06T13:51:27.044 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T13:51:27.044 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-06T13:51:27.044 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T13:51:27.044 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T13:51:27.571 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2023-10-06T13:51:27.573 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2023-10-06T13:51:27.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T13:51:27.701 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T13:51:27.739 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T13:51:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:27.844+0000 7f64b261d500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T13:51:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:27.845+0000 7f64b261d500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T13:51:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:27.845+0000 7f64b261d500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T13:51:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:27.845+0000 7f64b261d500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T13:51:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:27.845+0000 7f64b261d500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T13:51:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:27.845+0000 7f64b261d500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T13:51:30.689 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T13:51:30.724 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T13:51:30.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:30.818+0000 7f8b2740e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T13:51:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:30.819+0000 7f8b2740e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T13:51:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:30.819+0000 7f8b2740e500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T13:51:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:30.819+0000 7f8b2740e500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T13:51:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:30.819+0000 7f8b2740e500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T13:51:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:30.819+0000 7f8b2740e500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T13:51:33.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T13:51:33.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T13:51:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:33.792+0000 7f537d011500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T13:51:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:33.792+0000 7f537d011500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T13:51:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:33.792+0000 7f537d011500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T13:51:33.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:33.792+0000 7f537d011500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T13:51:33.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:33.792+0000 7f537d011500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T13:51:33.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:33.793+0000 7f537d011500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T13:51:36.648 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T13:51:36.685 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T13:51:36.778 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:36.776+0000 7f65e2021500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T13:51:36.778 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:36.776+0000 7f65e2021500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T13:51:36.778 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:36.776+0000 7f65e2021500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T13:51:36.778 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:36.776+0000 7f65e2021500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T13:51:36.779 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:36.776+0000 7f65e2021500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T13:51:36.779 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:36.777+0000 7f65e2021500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T13:51:39.627 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T13:51:39.663 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T13:51:39.694 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T13:51:39.695 INFO:tasks.ceph:role: osd.4 2023-10-06T13:51:39.695 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com 2023-10-06T13:51:39.695 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T13:51:39.772 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T13:51:39.772 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T13:51:39.772 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T13:51:39.772 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-06T13:51:39.772 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T13:51:39.773 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2023-10-06T13:51:39.773 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T13:51:39.773 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-06T13:51:39.773 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T13:51:39.773 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T13:51:40.279 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2023-10-06T13:51:40.281 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2023-10-06T13:51:40.281 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T13:51:40.431 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T13:51:40.467 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T13:51:40.536 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T13:51:40.536 INFO:tasks.ceph:role: osd.5 2023-10-06T13:51:40.536 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com 2023-10-06T13:51:40.536 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T13:51:40.612 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T13:51:40.612 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T13:51:40.612 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T13:51:40.612 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-06T13:51:40.612 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T13:51:40.613 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2023-10-06T13:51:40.613 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T13:51:40.613 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-06T13:51:40.613 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T13:51:40.613 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T13:51:41.064 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2023-10-06T13:51:41.066 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2023-10-06T13:51:41.066 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T13:51:41.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T13:51:41.197 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T13:51:41.266 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T13:51:41.266 INFO:tasks.ceph:role: osd.6 2023-10-06T13:51:41.266 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com 2023-10-06T13:51:41.266 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T13:51:41.341 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T13:51:41.341 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T13:51:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T13:51:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-06T13:51:41.342 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T13:51:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2023-10-06T13:51:41.342 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T13:51:41.342 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-06T13:51:41.342 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T13:51:41.342 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T13:51:41.828 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2023-10-06T13:51:41.829 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2023-10-06T13:51:41.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T13:51:41.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T13:51:42.022 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T13:51:42.092 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T13:51:42.092 INFO:tasks.ceph:role: osd.7 2023-10-06T13:51:42.092 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi083.front.sepia.ceph.com 2023-10-06T13:51:42.092 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T13:51:42.168 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T13:51:42.168 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T13:51:42.168 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T13:51:42.168 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-06T13:51:42.169 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T13:51:42.169 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2023-10-06T13:51:42.169 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T13:51:42.169 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-06T13:51:42.169 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T13:51:42.169 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T13:51:42.678 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2023-10-06T13:51:42.680 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2023-10-06T13:51:42.680 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T13:51:42.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T13:51:42.832 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T13:51:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:42.933+0000 7f85f8025500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T13:51:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:42.933+0000 7f85f8025500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T13:51:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:42.933+0000 7f85f8025500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T13:51:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:42.933+0000 7f85f8025500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T13:51:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:42.933+0000 7f85f8025500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T13:51:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:42.934+0000 7f85f8025500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T13:51:45.778 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T13:51:45.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T13:51:45.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:45.894+0000 7fe851ad8500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T13:51:45.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:45.895+0000 7fe851ad8500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T13:51:45.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:45.895+0000 7fe851ad8500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T13:51:45.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:45.895+0000 7fe851ad8500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T13:51:45.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:45.895+0000 7fe851ad8500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T13:51:45.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:45.895+0000 7fe851ad8500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T13:51:48.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T13:51:48.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T13:51:48.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:48.865+0000 7f8aff737500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T13:51:48.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:48.865+0000 7f8aff737500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T13:51:48.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:48.865+0000 7f8aff737500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T13:51:48.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:48.865+0000 7f8aff737500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T13:51:48.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:48.865+0000 7f8aff737500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T13:51:48.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:48.866+0000 7f8aff737500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T13:51:51.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T13:51:51.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T13:51:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:51.834+0000 7f9954ad8500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T13:51:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:51.834+0000 7f9954ad8500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T13:51:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:51.834+0000 7f9954ad8500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T13:51:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:51.834+0000 7f9954ad8500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T13:51:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:51.834+0000 7f9954ad8500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T13:51:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:51:51.835+0000 7f9954ad8500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T13:51:54.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T13:51:54.710 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T13:51:54.710 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:54.710 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T13:51:54.747 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:54.747 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T13:51:54.822 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:54.822 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T13:51:54.895 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:54.895 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T13:51:54.967 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:54.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T13:51:55.040 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:51:55.040 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T13:51:55.071 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:51:55.071 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T13:51:55.142 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:51:55.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T13:51:55.212 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:51:55.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T13:51:55.282 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:51:55.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T13:51:55.351 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:51:55.351 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T13:51:55.369 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:51:55.369 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T13:51:55.407 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T13:51:55.407 DEBUG:teuthology.orchestra.run.smithi019:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T13:51:55.411 DEBUG:teuthology.orchestra.run.smithi083:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T13:51:55.450 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr.y] 2023-10-06T13:51:55.450 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDcECBlXD5LERAAmmZjeNA+wyHJBnUEX3EA9Q== 2023-10-06T13:51:55.451 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.0] 2023-10-06T13:51:55.451 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDfECBlKU9rMhAAy9hr6DE+C6boDHMJTwDlNw== 2023-10-06T13:51:55.451 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.1] 2023-10-06T13:51:55.451 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDiECBlP/ndMBAAfVwjrzdOojn8a+zzUWrNOA== 2023-10-06T13:51:55.451 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.2] 2023-10-06T13:51:55.451 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDlECBlNJBHLxAAfnbrJYuiHj4DW28yx7BeWw== 2023-10-06T13:51:55.451 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.3] 2023-10-06T13:51:55.451 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDoECBlmQdVLhAAR4KhHmaVoG32kWPd/wBAtA== 2023-10-06T13:51:55.451 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr.x] 2023-10-06T13:51:55.451 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDcECBl+a1NFRAAiR3sQrlHbXkMwDcmqZFFyA== 2023-10-06T13:51:55.452 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.4] 2023-10-06T13:51:55.452 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDuECBlGC+pNxAALCcNdqBkaVb2XTf3CiK1zw== 2023-10-06T13:51:55.452 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.5] 2023-10-06T13:51:55.452 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDxECBlvRhhNRAAltOuTjcYemZBCgKcq0hZmw== 2023-10-06T13:51:55.452 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.6] 2023-10-06T13:51:55.452 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQD0ECBlaO6tMxAABvfwMLeESK/ttNdSDj6H4A== 2023-10-06T13:51:55.452 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.7] 2023-10-06T13:51:55.452 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQD3ECBl1h7RMRAANIA7nCulr8X0mWCGk9AP6Q== 2023-10-06T13:51:55.452 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2023-10-06T13:51:55.452 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDcECBl4nlGGBAANDroaIL4c6qozzoJZeyKEw== 2023-10-06T13:51:55.452 INFO:teuthology.orchestra.run.smithi019.stdout:[client.1] 2023-10-06T13:51:55.452 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDcECBliEalIBAASPyb/Dg+sb0caikgVIJtLw== 2023-10-06T13:51:55.473 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.y] 2023-10-06T13:51:55.473 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDcECBlXD5LERAAmmZjeNA+wyHJBnUEX3EA9Q== 2023-10-06T13:51:55.473 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.0] 2023-10-06T13:51:55.473 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDfECBlKU9rMhAAy9hr6DE+C6boDHMJTwDlNw== 2023-10-06T13:51:55.473 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.1] 2023-10-06T13:51:55.473 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDiECBlP/ndMBAAfVwjrzdOojn8a+zzUWrNOA== 2023-10-06T13:51:55.474 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.2] 2023-10-06T13:51:55.474 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDlECBlNJBHLxAAfnbrJYuiHj4DW28yx7BeWw== 2023-10-06T13:51:55.474 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.3] 2023-10-06T13:51:55.474 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDoECBlmQdVLhAAR4KhHmaVoG32kWPd/wBAtA== 2023-10-06T13:51:55.474 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.x] 2023-10-06T13:51:55.474 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDcECBl+a1NFRAAiR3sQrlHbXkMwDcmqZFFyA== 2023-10-06T13:51:55.474 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.4] 2023-10-06T13:51:55.474 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDuECBlGC+pNxAALCcNdqBkaVb2XTf3CiK1zw== 2023-10-06T13:51:55.474 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.5] 2023-10-06T13:51:55.474 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDxECBlvRhhNRAAltOuTjcYemZBCgKcq0hZmw== 2023-10-06T13:51:55.475 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.6] 2023-10-06T13:51:55.475 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQD0ECBlaO6tMxAABvfwMLeESK/ttNdSDj6H4A== 2023-10-06T13:51:55.475 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.7] 2023-10-06T13:51:55.475 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQD3ECBl1h7RMRAANIA7nCulr8X0mWCGk9AP6Q== 2023-10-06T13:51:55.475 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2023-10-06T13:51:55.475 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDcECBl4nlGGBAANDroaIL4c6qozzoJZeyKEw== 2023-10-06T13:51:55.475 INFO:teuthology.orchestra.run.smithi083.stdout:[client.1] 2023-10-06T13:51:55.475 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDcECBliEalIBAASPyb/Dg+sb0caikgVIJtLw== 2023-10-06T13:51:55.476 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T13:51:55.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T13:51:55.570 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:55.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:55.627 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:55.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:55.683 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:55.686 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:55.739 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:55.741 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:55.794 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T13:51:55.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T13:51:55.849 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:55.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:55.905 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:55.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:55.960 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:55.962 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:56.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:56.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T13:51:56.070 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T13:51:56.073 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T13:51:56.129 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T13:51:56.131 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T13:51:56.187 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T13:51:56.187 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T13:51:56.217 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T13:51:56.318 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T13:51:56.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T13:51:56.419 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T13:51:56.519 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T13:51:56.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T13:51:56.577 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T13:51:56.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T13:51:56.700 DEBUG:teuthology.orchestra.run.smithi019:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T13:51:56.703 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T13:51:56.754 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T13:51:56.755 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-10-06T13:51:56.755 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T13:51:56.755 DEBUG:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2023-10-06T13:51:56.758 INFO:tasks.ceph.mon.a:Started 2023-10-06T13:51:56.758 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-10-06T13:51:56.758 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T13:51:56.758 DEBUG:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2023-10-06T13:51:56.760 INFO:tasks.ceph.mon.c:Started 2023-10-06T13:51:56.760 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-10-06T13:51:56.760 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T13:51:56.761 DEBUG:teuthology.orchestra.run.smithi083:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2023-10-06T13:51:56.796 INFO:tasks.ceph.mon.b:Started 2023-10-06T13:51:56.796 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T13:51:56.796 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T13:51:56.797 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2023-10-06T13:51:56.799 INFO:tasks.ceph.mgr.y:Started 2023-10-06T13:51:56.799 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T13:51:56.799 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2023-10-06T13:51:56.801 INFO:tasks.ceph.mgr.x:Started 2023-10-06T13:51:56.801 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T13:51:56.802 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T13:51:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.906+0000 7f996d23a640 10 monclient: get_monmap_and_config 2023-10-06T13:51:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.906+0000 7f996d23a640 10 monclient: build_initial_monmap 2023-10-06T13:51:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.906+0000 7f996d23a640 10 monclient: monmap: 2023-10-06T13:51:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:51:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:51:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:56.907170+0000 2023-10-06T13:51:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:56.907170+0000 2023-10-06T13:51:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:51:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:51:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:51:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:51:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:51:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:51:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:51:56.909 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:51:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.906+0000 7f996d23a640 1 Processor -- start 2023-10-06T13:51:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.906+0000 7f996d23a640 1 -- start start 2023-10-06T13:51:56.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.906+0000 7f996d23a640 10 monclient: init 2023-10-06T13:51:56.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 10 monclient: _reopen_session rank -1 2023-10-06T13:51:56.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2023-10-06T13:51:56.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 0x7f99680af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:51:56.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 10 monclient(hunting): picked mon.noname-b con 0x7f99680b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:51:56.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f99680b3d60 0x7f9968141500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:51:56.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 10 monclient(hunting): picked mon.noname-a con 0x7f99680b3d60 addr v2:172.21.15.19:3300/0 2023-10-06T13:51:56.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f99680b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:51:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 10 monclient(hunting): picked mon.noname-c con 0x7f9968141a40 addr v2:172.21.15.83:3300/0 2023-10-06T13:51:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:51:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9968140dd0 con 0x7f99680b3d60 2023-10-06T13:51:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:51:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99680aebe0 con 0x7f9968141a40 2023-10-06T13:51:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:51:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9968141070 con 0x7f99680b3990 2023-10-06T13:51:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 10 monclient(hunting): _renew_subs 2023-10-06T13:51:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996d23a640 10 monclient(hunting): authenticate will time out at 1245.093797s 2023-10-06T13:51:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996659c640 1 -- >> v2:172.21.15.19:3300/0 conn(0x7f99680b3d60 msgr2=0x7f9968141500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3300/0 2023-10-06T13:51:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f9966d9d640 1 -- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 msgr2=0x7f99680af810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3301/0 2023-10-06T13:51:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f99680b3d60 0x7f9968141500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T13:51:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f9966d9d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 0x7f99680af810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T13:51:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996759e640 1 -- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f99680b00c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.83:3300/0 2023-10-06T13:51:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:56.907+0000 7f996759e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f99680b00c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T13:51:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:57.108+0000 7f996659c640 1 -- >> v2:172.21.15.19:3300/0 conn(0x7f99680b3d60 msgr2=0x7f9968141500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3300/0 2023-10-06T13:51:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:57.108+0000 7f9966d9d640 1 -- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 msgr2=0x7f99680af810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3301/0 2023-10-06T13:51:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:57.108+0000 7f996659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f99680b3d60 0x7f9968141500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T13:51:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:57.108+0000 7f9966d9d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 0x7f99680af810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T13:51:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:57.108+0000 7f996759e640 1 -- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f99680b00c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.83:3300/0 2023-10-06T13:51:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:57.108+0000 7f996759e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f99680b00c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T13:51:57.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:57.509+0000 7f9966d9d640 1 -- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 msgr2=0x7f99680af810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3301/0 2023-10-06T13:51:57.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:57.509+0000 7f996659c640 1 -- >> v2:172.21.15.19:3300/0 conn(0x7f99680b3d60 msgr2=0x7f9968141500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3300/0 2023-10-06T13:51:57.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:57.509+0000 7f996659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f99680b3d60 0x7f9968141500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T13:51:57.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:57.509+0000 7f9966d9d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 0x7f99680af810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T13:51:57.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:57.509+0000 7f996759e640 1 -- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f99680b00c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.83:3300/0 2023-10-06T13:51:57.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:57.509+0000 7f996759e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f99680b00c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T13:51:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:58.310+0000 7f9966d9d640 1 -- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 msgr2=0x7f99680af810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3301/0 2023-10-06T13:51:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:58.310+0000 7f996659c640 1 -- >> v2:172.21.15.19:3300/0 conn(0x7f99680b3d60 msgr2=0x7f9968141500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3300/0 2023-10-06T13:51:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:58.310+0000 7f996659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f99680b3d60 0x7f9968141500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T13:51:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:58.310+0000 7f9966d9d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 0x7f99680af810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T13:51:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:58.310+0000 7f996759e640 1 -- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f99680b00c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.83:3300/0 2023-10-06T13:51:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:58.310+0000 7f996759e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f99680b00c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T13:51:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 10 monclient(hunting): tick 2023-10-06T13:51:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 10 monclient(hunting): _check_auth_tickets 2023-10-06T13:51:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 1 monclient(hunting): continuing hunt 2023-10-06T13:51:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 10 monclient(hunting): _reopen_session rank -1 2023-10-06T13:51:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 1 -- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 msgr2=0x7f99680af810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:51:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 0x7f99680af810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:51:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 1 -- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f99680b00c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:51:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f99680b00c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:51:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 1 -- >> v2:172.21.15.19:3300/0 conn(0x7f99680b3d60 msgr2=0x7f9968141500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:51:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f99680b3d60 0x7f9968141500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:51:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 10 monclient: _add_conns ranks=[1,0] 2023-10-06T13:51:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f99540036b0 addr v1:172.21.15.83:6789/0 2023-10-06T13:51:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9954007350 addr v1:172.21.15.19:6789/0 2023-10-06T13:51:59.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99680aee30 con 0x7f9954007350 2023-10-06T13:51:59.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995400c0c0 con 0x7f99540036b0 2023-10-06T13:51:59.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9964d99640 10 monclient(hunting): _renew_subs 2023-10-06T13:51:59.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f996659c640 1 -- >> v1:172.21.15.19:6789/0 conn(0x7f9954007350 legacy=0x7f9954009800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.19:6789/0 2023-10-06T13:51:59.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:51:59.907+0000 7f9966d9d640 1 -- >> v1:172.21.15.83:6789/0 conn(0x7f99540036b0 legacy=0x7f9954005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.83:6789/0 2023-10-06T13:52:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:00.108+0000 7f996659c640 1 -- >> v1:172.21.15.19:6789/0 conn(0x7f9954007350 legacy=0x7f9954009800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.19:6789/0 2023-10-06T13:52:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:00.108+0000 7f9966d9d640 1 -- >> v1:172.21.15.83:6789/0 conn(0x7f99540036b0 legacy=0x7f9954005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.83:6789/0 2023-10-06T13:52:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:00.509+0000 7f996659c640 1 -- >> v1:172.21.15.19:6789/0 conn(0x7f9954007350 legacy=0x7f9954009800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.19:6789/0 2023-10-06T13:52:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:00.509+0000 7f9966d9d640 1 -- >> v1:172.21.15.83:6789/0 conn(0x7f99540036b0 legacy=0x7f9954005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.83:6789/0 2023-10-06T13:52:01.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:01.310+0000 7f996659c640 1 -- >> v1:172.21.15.19:6789/0 conn(0x7f9954007350 legacy=0x7f9954009800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.19:6789/0 2023-10-06T13:52:01.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:01.310+0000 7f9966d9d640 1 -- >> v1:172.21.15.83:6789/0 conn(0x7f99540036b0 legacy=0x7f9954005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.83:6789/0 2023-10-06T13:52:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 10 monclient(hunting): tick 2023-10-06T13:52:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 10 monclient(hunting): _check_auth_tickets 2023-10-06T13:52:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 1 monclient(hunting): continuing hunt 2023-10-06T13:52:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 10 monclient(hunting): _reopen_session rank -1 2023-10-06T13:52:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 1 -- >> v1:172.21.15.83:6789/0 conn(0x7f99540036b0 legacy=0x7f9954005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:52:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 1 -- >> v1:172.21.15.19:6789/0 conn(0x7f9954007350 legacy=0x7f9954009800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:52:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 10 monclient: _add_conns ranks=[4,1,2,0,3] 2023-10-06T13:52:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f996759e640 1 -- reap_dead start 2023-10-06T13:52:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-c con 0x7f9968141a40 addr v2:172.21.15.83:3300/0 2023-10-06T13:52:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f99540036b0 addr v1:172.21.15.83:6789/0 2023-10-06T13:52:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 0x7f995400c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-b con 0x7f99680b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995400d320 con 0x7f99540036b0 2023-10-06T13:52:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9968140dd0 con 0x7f9968141a40 2023-10-06T13:52:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99680aebe0 con 0x7f99680b3990 2023-10-06T13:52:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9964d99640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:02.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9966d9d640 1 -- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.83:3300/0 2023-10-06T13:52:02.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f996759e640 1 -- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 msgr2=0x7f995400c840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3301/0 2023-10-06T13:52:02.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f9966d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T13:52:02.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f996759e640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 0x7f995400c840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T13:52:02.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:02.907+0000 7f996659c640 1 -- >> v1:172.21.15.83:6789/0 conn(0x7f99540036b0 legacy=0x7f9954005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.83:6789/0 2023-10-06T13:52:03.110 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:03.108+0000 7f996759e640 1 -- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 msgr2=0x7f995400c840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3301/0 2023-10-06T13:52:03.110 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:03.108+0000 7f996759e640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 0x7f995400c840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T13:52:03.110 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:03.108+0000 7f9966d9d640 1 -- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.83:3300/0 2023-10-06T13:52:03.110 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:03.108+0000 7f996659c640 1 -- >> v1:172.21.15.83:6789/0 conn(0x7f99540036b0 legacy=0x7f9954005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.83:6789/0 2023-10-06T13:52:03.110 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:03.108+0000 7f9966d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T13:52:03.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:03.509+0000 7f996759e640 1 -- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 msgr2=0x7f995400c840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3301/0 2023-10-06T13:52:03.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:03.509+0000 7f996759e640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 0x7f995400c840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T13:52:03.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:03.509+0000 7f996659c640 1 -- >> v1:172.21.15.83:6789/0 conn(0x7f99540036b0 legacy=0x7f9954005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.83:6789/0 2023-10-06T13:52:03.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:03.509+0000 7f9966d9d640 1 -- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.83:3300/0 2023-10-06T13:52:03.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:03.509+0000 7f9966d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T13:52:04.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:04.310+0000 7f996759e640 1 -- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 msgr2=0x7f995400c840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3301/0 2023-10-06T13:52:04.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:04.310+0000 7f996759e640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 0x7f995400c840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T13:52:04.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:04.310+0000 7f9966d9d640 1 -- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.83:3300/0 2023-10-06T13:52:04.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:04.310+0000 7f996659c640 1 -- >> v1:172.21.15.83:6789/0 conn(0x7f99540036b0 legacy=0x7f9954005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.83:6789/0 2023-10-06T13:52:04.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:04.310+0000 7f9966d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T13:52:04.434 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:00:07.531 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:52:04.746 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:00:07.843 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:52:04.991 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:00:08.114 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:52:04.997 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:00:08.120 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:52:05.324 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:00:08.447 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:52:05.330 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:00:08.453 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:52:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 10 monclient(hunting): tick 2023-10-06T13:52:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 10 monclient(hunting): _check_auth_tickets 2023-10-06T13:52:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 1 monclient(hunting): continuing hunt 2023-10-06T13:52:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 10 monclient(hunting): _reopen_session rank -1 2023-10-06T13:52:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 1 -- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 msgr2=0x7f995400c840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:52:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99680b3990 0x7f995400c840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 1 -- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:52:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 1 -- >> v1:172.21.15.83:6789/0 conn(0x7f99540036b0 legacy=0x7f9954005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:52:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 10 monclient: _add_conns ranks=[0,3] 2023-10-06T13:52:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f995400b070 addr v1:172.21.15.19:6789/0 2023-10-06T13:52:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 0x7f995400b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-a con 0x7f995400b440 addr v2:172.21.15.19:3300/0 2023-10-06T13:52:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99680aee30 con 0x7f995400b070 2023-10-06T13:52:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9968140dd0 con 0x7f995400b440 2023-10-06T13:52:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9964d99640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f996659c640 1 -- >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 msgr2=0x7f995400b810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3300/0 2023-10-06T13:52:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.907+0000 7f9966d9d640 1 -- >> v1:172.21.15.19:6789/0 conn(0x7f995400b070 legacy=0x7f9954009800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.19:6789/0 2023-10-06T13:52:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:05.908+0000 7f996659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 0x7f995400b810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T13:52:06.110 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:06.109+0000 7f996659c640 1 -- >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 msgr2=0x7f995400b810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3300/0 2023-10-06T13:52:06.111 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:06.109+0000 7f9966d9d640 1 -- >> v1:172.21.15.19:6789/0 conn(0x7f995400b070 legacy=0x7f9954009800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.19:6789/0 2023-10-06T13:52:06.111 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:06.109+0000 7f996659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 0x7f995400b810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T13:52:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:06.510+0000 7f9966d9d640 1 -- >> v1:172.21.15.19:6789/0 conn(0x7f995400b070 legacy=0x7f9954009800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.19:6789/0 2023-10-06T13:52:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:06.510+0000 7f996659c640 1 -- >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 msgr2=0x7f995400b810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:3300/0 2023-10-06T13:52:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:06.510+0000 7f996659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 0x7f995400b810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T13:52:07.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:07.478+0000 7f996659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 0x7f995400b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:07.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:07.478+0000 7f996659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 0x7f995400b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46380/0 (socket says 172.21.15.19:46380) 2023-10-06T13:52:07.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:07.478+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 learned_addr learned my addr 172.21.15.19:0/983106925 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:52:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:07.478+0000 7f996659c640 10 monclient(hunting): get_auth_request con 0x7f995400b440 auth_method 0 2023-10-06T13:52:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:07.478+0000 7f996659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:07.478+0000 7f996659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:07.478+0000 7f996659c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:07.644+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 msgr2=0x7f995400b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:07.644+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 msgr2=0x7f995400b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:07.644+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 0x7f995400b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:07.644+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 0x7f995400b810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T13:52:08.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 10 monclient(hunting): tick 2023-10-06T13:52:08.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 10 monclient(hunting): _check_auth_tickets 2023-10-06T13:52:08.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 1 monclient(hunting): continuing hunt 2023-10-06T13:52:08.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 10 monclient(hunting): _reopen_session rank -1 2023-10-06T13:52:08.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 msgr2=0x7f995400b810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:52:08.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b440 0x7f995400b810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:08.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 >> v1:172.21.15.19:6789/0 conn(0x7f995400b070 legacy=0x7f9954009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:08.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 10 monclient: _add_conns ranks=[4,1,2,0,3] 2023-10-06T13:52:08.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f996759e640 1 -- 172.21.15.19:0/983106925 reap_dead start 2023-10-06T13:52:08.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:08.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-c con 0x7f9968141a40 addr v2:172.21.15.83:3300/0 2023-10-06T13:52:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f99540036b0 addr v1:172.21.15.83:6789/0 2023-10-06T13:52:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 0x7f9954009800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-b con 0x7f995400c840 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995400d320 con 0x7f99540036b0 2023-10-06T13:52:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9968140dd0 con 0x7f9968141a40 2023-10-06T13:52:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99680aebe0 con 0x7f995400c840 2023-10-06T13:52:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9964d99640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 0x7f9954009800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.908+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.909+0000 7f996759e640 10 monclient(hunting): get_auth_request con 0x7f995400c840 auth_method 0 2023-10-06T13:52:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.909+0000 7f996759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.909+0000 7f996759e640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.909+0000 7f996759e640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.909+0000 7f9966d9d640 10 monclient(hunting): get_auth_request con 0x7f9968141a40 auth_method 0 2023-10-06T13:52:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.909+0000 7f9966d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.909+0000 7f9966d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.909+0000 7f9966d9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.910+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.910+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.910+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.910+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T13:52:08.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.916+0000 7f996759e640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 msgr2=0x7f9954009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-06T13:52:08.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.916+0000 7f996759e640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 msgr2=0x7f9954009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:08.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.916+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 0x7f9954009800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:08.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:08.916+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 0x7f9954009800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T13:52:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.112+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.112+0000 7f9966d9d640 10 monclient(hunting): get_auth_request con 0x7f9968141a40 auth_method 0 2023-10-06T13:52:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.112+0000 7f9966d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.112+0000 7f9966d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.112+0000 7f9966d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.112+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.112+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.112+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.112+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T13:52:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.117+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 0x7f9954009800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.117+0000 7f996759e640 10 monclient(hunting): get_auth_request con 0x7f995400c840 auth_method 0 2023-10-06T13:52:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.117+0000 7f996759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.117+0000 7f996759e640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.117+0000 7f996759e640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.118+0000 7f996759e640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 msgr2=0x7f9954009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.118+0000 7f996759e640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 msgr2=0x7f9954009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.118+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 0x7f9954009800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:09.121 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.118+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 0x7f9954009800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T13:52:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.514+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.514+0000 7f9966d9d640 10 monclient(hunting): get_auth_request con 0x7f9968141a40 auth_method 0 2023-10-06T13:52:09.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.514+0000 7f9966d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.514+0000 7f9966d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.514+0000 7f9966d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.515+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.515+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.515+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.515+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T13:52:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.519+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 0x7f9954009800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.520+0000 7f996759e640 10 monclient(hunting): get_auth_request con 0x7f995400c840 auth_method 0 2023-10-06T13:52:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.520+0000 7f996759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.520+0000 7f996759e640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.520+0000 7f996759e640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.521+0000 7f996759e640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 msgr2=0x7f9954009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.521+0000 7f996759e640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 msgr2=0x7f9954009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.521+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 0x7f9954009800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:09.521+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 0x7f9954009800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T13:52:10.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.316+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:10.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.317+0000 7f9966d9d640 10 monclient(hunting): get_auth_request con 0x7f9968141a40 auth_method 0 2023-10-06T13:52:10.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.317+0000 7f9966d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:10.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.317+0000 7f9966d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:10.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.317+0000 7f9966d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.318+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.318+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.318+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.318+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T13:52:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.323+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 0x7f9954009800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.323+0000 7f996759e640 10 monclient(hunting): get_auth_request con 0x7f995400c840 auth_method 0 2023-10-06T13:52:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.323+0000 7f996759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.323+0000 7f996759e640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.323+0000 7f996759e640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:10.326 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.324+0000 7f996759e640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 msgr2=0x7f9954009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:10.326 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.324+0000 7f996759e640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 msgr2=0x7f9954009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:10.326 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.324+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 0x7f9954009800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:10.326 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:10.324+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 0x7f9954009800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T13:52:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 10 monclient(hunting): tick 2023-10-06T13:52:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 10 monclient(hunting): _check_auth_tickets 2023-10-06T13:52:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 1 monclient(hunting): continuing hunt 2023-10-06T13:52:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 10 monclient(hunting): _reopen_session rank -1 2023-10-06T13:52:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 msgr2=0x7f9954009800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:52:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f995400c840 0x7f9954009800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:52:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 >> v1:172.21.15.83:6789/0 conn(0x7f99540036b0 legacy=0x7f9954005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 10 monclient: _add_conns ranks=[3,0] 2023-10-06T13:52:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-a con 0x7f995400b810 addr v2:172.21.15.19:3300/0 2023-10-06T13:52:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f995400bbe0 addr v1:172.21.15.19:6789/0 2023-10-06T13:52:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995400c0c0 con 0x7f995400bbe0 2023-10-06T13:52:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:11.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9968140dd0 con 0x7f995400b810 2023-10-06T13:52:11.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.908+0000 7f9964d99640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:11.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.909+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:11.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.909+0000 7f9966d9d640 10 monclient(hunting): get_auth_request con 0x7f995400b810 auth_method 0 2023-10-06T13:52:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.909+0000 7f9966d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.909+0000 7f9966d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.909+0000 7f9966d9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.910+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T13:52:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.910+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.910+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:11.910+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T13:52:12.113 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.111+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:12.113 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.111+0000 7f9966d9d640 10 monclient(hunting): get_auth_request con 0x7f995400b810 auth_method 0 2023-10-06T13:52:12.114 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.111+0000 7f9966d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:12.114 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.111+0000 7f9966d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:12.114 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.111+0000 7f9966d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:12.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.113+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T13:52:12.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.113+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:12.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.113+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:12.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.113+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T13:52:12.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.514+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:12.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.515+0000 7f9966d9d640 10 monclient(hunting): get_auth_request con 0x7f995400b810 auth_method 0 2023-10-06T13:52:12.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.515+0000 7f9966d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:12.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.515+0000 7f9966d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:12.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.515+0000 7f9966d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:12.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.515+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T13:52:12.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.515+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:12.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.515+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:12.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:12.515+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T13:52:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:13.898+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:13.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:13.901+0000 7f9966d9d640 10 monclient(hunting): get_auth_request con 0x7f995400b810 auth_method 0 2023-10-06T13:52:13.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:13.901+0000 7f9966d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:13.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:13.901+0000 7f9966d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:13.901+0000 7f9966d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:13.917+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T13:52:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:13.917+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:13.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:13.917+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:13.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:13.917+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T13:52:14.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 10 monclient(hunting): tick 2023-10-06T13:52:14.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 10 monclient(hunting): _check_auth_tickets 2023-10-06T13:52:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 1 monclient(hunting): continuing hunt 2023-10-06T13:52:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 10 monclient(hunting): _reopen_session rank -1 2023-10-06T13:52:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:52:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 >> v1:172.21.15.19:6789/0 conn(0x7f995400bbe0 legacy=0x7f995400a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 10 monclient: _add_conns ranks=[3,4,1,0,2] 2023-10-06T13:52:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f996759e640 1 -- 172.21.15.19:0/983106925 reap_dead start 2023-10-06T13:52:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-a con 0x7f995400b810 addr v2:172.21.15.19:3300/0 2023-10-06T13:52:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-c con 0x7f9968141a40 addr v2:172.21.15.83:3300/0 2023-10-06T13:52:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f99540036b0 addr v1:172.21.15.83:6789/0 2023-10-06T13:52:14.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995400acd0 con 0x7f99540036b0 2023-10-06T13:52:14.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:14.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9968140dd0 con 0x7f995400b810 2023-10-06T13:52:14.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:14.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99680aebe0 con 0x7f9968141a40 2023-10-06T13:52:14.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.908+0000 7f9964d99640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:14.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.910+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:14.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.910+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:14.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.911+0000 7f9966d9d640 10 monclient(hunting): get_auth_request con 0x7f995400b810 auth_method 0 2023-10-06T13:52:14.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.911+0000 7f9966d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:14.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.911+0000 7f9966d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:14.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.911+0000 7f9966d9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:14.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.911+0000 7f996659c640 10 monclient(hunting): get_auth_request con 0x7f9968141a40 auth_method 0 2023-10-06T13:52:14.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.911+0000 7f996659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:14.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.911+0000 7f996659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:14.913 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.911+0000 7f996659c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:14.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.912+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:14.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.912+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:14.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.912+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:14.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.912+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T13:52:14.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.912+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:14.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.912+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:14.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.912+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T13:52:14.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:14.912+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T13:52:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.113+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.113+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.113+0000 7f996659c640 10 monclient(hunting): get_auth_request con 0x7f9968141a40 auth_method 0 2023-10-06T13:52:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.113+0000 7f996659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.113+0000 7f996659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.113+0000 7f996659c640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:15.116 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.113+0000 7f9966d9d640 10 monclient(hunting): get_auth_request con 0x7f995400b810 auth_method 0 2023-10-06T13:52:15.116 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.113+0000 7f9966d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:15.116 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.113+0000 7f9966d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:15.116 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.113+0000 7f9966d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:15.116 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.114+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.114+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.114+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.114+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T13:52:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.114+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T13:52:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.114+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.114+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.114+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T13:52:15.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.515+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:15.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.515+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:15.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.515+0000 7f996659c640 10 monclient(hunting): get_auth_request con 0x7f9968141a40 auth_method 0 2023-10-06T13:52:15.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.515+0000 7f996659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:15.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.515+0000 7f996659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:15.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.515+0000 7f996659c640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:15.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.515+0000 7f9966d9d640 10 monclient(hunting): get_auth_request con 0x7f995400b810 auth_method 0 2023-10-06T13:52:15.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.515+0000 7f9966d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:15.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.515+0000 7f9966d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:15.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.515+0000 7f9966d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:15.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.516+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T13:52:15.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.516+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:15.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.516+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:15.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.516+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T13:52:15.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.516+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:15.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.516+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:15.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.516+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:15.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:15.516+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T13:52:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.318+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.318+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.318+0000 7f996659c640 10 monclient(hunting): get_auth_request con 0x7f9968141a40 auth_method 0 2023-10-06T13:52:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.318+0000 7f996659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.318+0000 7f996659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.318+0000 7f996659c640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.318+0000 7f9966d9d640 10 monclient(hunting): get_auth_request con 0x7f995400b810 auth_method 0 2023-10-06T13:52:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.318+0000 7f9966d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.318+0000 7f9966d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.318+0000 7f9966d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.318+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T13:52:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.318+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.318+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.318+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T13:52:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.319+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.319+0000 7f9966d9d640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.319+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:16.319+0000 7f9966d9d640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T13:52:17.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 10 monclient(hunting): tick 2023-10-06T13:52:17.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 10 monclient(hunting): _check_auth_tickets 2023-10-06T13:52:17.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 1 monclient(hunting): continuing hunt 2023-10-06T13:52:17.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 10 monclient(hunting): _reopen_session rank -1 2023-10-06T13:52:17.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 msgr2=0x7f995400c280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:52:17.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.83:3300/0 conn(0x7f9968141a40 0x7f995400c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:17.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 msgr2=0x7f9954009fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:52:17.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 1 --2- 172.21.15.19:0/983106925 >> v2:172.21.15.19:3300/0 conn(0x7f995400b810 0x7f9954009fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:17.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 >> v1:172.21.15.83:6789/0 conn(0x7f99540036b0 legacy=0x7f9954005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:17.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 10 monclient: _add_conns ranks=[0,2] 2023-10-06T13:52:17.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f995400a680 addr v1:172.21.15.19:6789/0 2023-10-06T13:52:17.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f9954014d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:17.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 10 monclient(hunting): picked mon.noname-b con 0x7f99540097c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:17.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995400a4f0 con 0x7f995400a680 2023-10-06T13:52:17.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:17.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 1 -- 172.21.15.19:0/983106925 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9968140dd0 con 0x7f99540097c0 2023-10-06T13:52:17.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f9964d99640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:17.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f9954014d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:17.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f996659c640 10 monclient(hunting): get_auth_request con 0x7f99540097c0 auth_method 0 2023-10-06T13:52:17.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f996659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:17.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f996659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:17.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.909+0000 7f996659c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:17.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.910+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 msgr2=0x7f9954014d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:17.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.910+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 msgr2=0x7f9954014d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:17.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.910+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f9954014d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:17.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:17.910+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f9954014d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T13:52:18.113 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.111+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f9954014d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:18.114 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.112+0000 7f996659c640 10 monclient(hunting): get_auth_request con 0x7f99540097c0 auth_method 0 2023-10-06T13:52:18.114 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.112+0000 7f996659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:18.114 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.112+0000 7f996659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:18.114 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.112+0000 7f996659c640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:18.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.112+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 msgr2=0x7f9954014d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:18.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.112+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 msgr2=0x7f9954014d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:18.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.112+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f9954014d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:18.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.113+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f9954014d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T13:52:18.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.514+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f9954014d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:18.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.514+0000 7f996659c640 10 monclient(hunting): get_auth_request con 0x7f99540097c0 auth_method 0 2023-10-06T13:52:18.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.514+0000 7f996659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:18.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.514+0000 7f996659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:18.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.514+0000 7f996659c640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:18.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.515+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 msgr2=0x7f9954014d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:18.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.515+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 msgr2=0x7f9954014d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:18.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.515+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f9954014d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:18.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:18.515+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f9954014d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T13:52:19.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:19.320+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f9954014d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:19.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:19.320+0000 7f996659c640 10 monclient(hunting): get_auth_request con 0x7f99540097c0 auth_method 0 2023-10-06T13:52:19.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:19.320+0000 7f996659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:19.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:19.320+0000 7f996659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:19.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:19.320+0000 7f996659c640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:19.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:19.321+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 msgr2=0x7f9954014d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T13:52:19.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:19.321+0000 7f996659c640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 msgr2=0x7f9954014d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T13:52:19.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:19.321+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f9954014d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T13:52:19.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:19.321+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f9954014d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T13:52:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.356+0000 7f9965d9b640 1 -- 172.21.15.19:0/983106925 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573314568 0 0) 0x7f995400a4f0 con 0x7f995400a680 2023-10-06T13:52:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.356+0000 7f9965d9b640 10 monclient(hunting): _negotiate 2023-10-06T13:52:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.356+0000 7f9965d9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.356+0000 7f9965d9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.356+0000 7f9965d9b640 10 monclient(hunting): my global_id is 4101 2023-10-06T13:52:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.356+0000 7f9965d9b640 1 -- 172.21.15.19:0/983106925 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9950003520 con 0x7f995400a680 2023-10-06T13:52:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 1 -- 172.21.15.19:0/983106925 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3451953409 0 0) 0x7f9950003520 con 0x7f995400a680 2023-10-06T13:52:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 msgr2=0x7f9954014d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T13:52:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f9954014d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient: _finish_hunting 0 2023-10-06T13:52:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 1 monclient: found mon.noname-a-legacy 2023-10-06T13:52:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.19:6789/0 2023-10-06T13:52:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 1 -- 172.21.15.19:0/983106925 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99540162b0 con 0x7f995400a680 2023-10-06T13:52:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient: _finish_auth 0 2023-10-06T13:52:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient: _check_auth_tickets 2023-10-06T13:52:20.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:20.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 1 -- 172.21.15.19:0/983106925 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251531364 0 0) 0x7f9958009810 con 0x7f995400a680 2023-10-06T13:52:20.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:20.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2023-10-06T13:52:20.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient: dump: 2023-10-06T13:52:20.422 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:52:20.422 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:20.422 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:20.422 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:20.422 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:20.422 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:20.423 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:52:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] but i'm connected to v1:172.21.15.19:6789/0, reconnecting 2023-10-06T13:52:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 1 -- 172.21.15.19:0/983106925 >> v1:172.21.15.19:6789/0 conn(0x7f995400a680 legacy=0x7f9954011630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:52:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f996759e640 1 -- 172.21.15.19:0/983106925 reap_dead start 2023-10-06T13:52:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9950003970 0x7f9950003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient(hunting): picked mon.a con 0x7f9950003970 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f99500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient(hunting): picked mon.c con 0x7f99540097c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9950004d00 0x7f9950005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient(hunting): picked mon.b con 0x7f9950004d00 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 1 -- 172.21.15.19:0/983106925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9968140dd0 con 0x7f9950003970 2023-10-06T13:52:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 1 -- 172.21.15.19:0/983106925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995000bb40 con 0x7f9950004d00 2023-10-06T13:52:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 1 -- 172.21.15.19:0/983106925 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995000bd00 con 0x7f99540097c0 2023-10-06T13:52:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f9965d9b640 10 monclient(hunting): _finish_auth 0 2023-10-06T13:52:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9950004d00 0x7f9950005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.418+0000 7f996659c640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f99500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.419+0000 7f996759e640 10 monclient(hunting): get_auth_request con 0x7f9950004d00 auth_method 0 2023-10-06T13:52:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.419+0000 7f996759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.419+0000 7f996759e640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.419+0000 7f996759e640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.419+0000 7f996659c640 10 monclient(hunting): get_auth_request con 0x7f99540097c0 auth_method 0 2023-10-06T13:52:20.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.419+0000 7f996659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.419+0000 7f996659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:20.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.419+0000 7f996659c640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.426+0000 7f996759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.426+0000 7f996759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:20.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.426+0000 7f996759e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T13:52:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.426+0000 7f996659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.426+0000 7f996659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.426+0000 7f996659c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T13:52:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996759e640 10 monclient(hunting): handle_auth_done global_id 4101 payload 293 2023-10-06T13:52:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996759e640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 msgr2=0x7f99500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f99500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996759e640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9950003970 msgr2=0x7f9950003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9950003970 0x7f9950003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996759e640 10 monclient: _finish_hunting 0 2023-10-06T13:52:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996759e640 1 monclient: found mon.b 2023-10-06T13:52:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996759e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996759e640 1 -- 172.21.15.19:0/983106925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9950003520 con 0x7f9950004d00 2023-10-06T13:52:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996759e640 10 monclient: _finish_auth 0 2023-10-06T13:52:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996759e640 10 monclient: _check_auth_tickets 2023-10-06T13:52:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996759e640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996d23a640 5 monclient: authenticate success, global_id 4101 2023-10-06T13:52:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.469+0000 7f996d23a640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:52:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.473+0000 7f996759e640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9950004d00 0x7f9950005170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f996806a5f0 tx=0x7f99500057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c988b10034198936 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.518+0000 7f9965d9b640 1 -- 172.21.15.19:0/983106925 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996000d040 con 0x7f9950004d00 2023-10-06T13:52:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.518+0000 7f9965d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.518+0000 7f9965d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.518+0000 7f9965d9b640 10 monclient: dump: 2023-10-06T13:52:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:52:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:20.521 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:52:20.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.518+0000 7f9965d9b640 1 -- 172.21.15.19:0/983106925 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9960030070 con 0x7f9950004d00 2023-10-06T13:52:20.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.518+0000 7f996d23a640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:52:20.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.518+0000 7f9965d9b640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:20.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.518+0000 7f9965d9b640 1 -- 172.21.15.19:0/983106925 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996002cc70 con 0x7f9950004d00 2023-10-06T13:52:20.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.518+0000 7f996d23a640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:20.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.518+0000 7f996d23a640 4 set_mon_vals no callback set 2023-10-06T13:52:20.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.518+0000 7f9965d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:20.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.518+0000 7f9965d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:20.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.518+0000 7f9965d9b640 10 monclient: dump: 2023-10-06T13:52:20.522 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:52:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:20.523 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:52:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.519+0000 7f996d23a640 10 monclient: shutdown 2023-10-06T13:52:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.519+0000 7f996d23a640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.519+0000 7f996d23a640 1 -- 172.21.15.19:0/983106925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9950004d00 msgr2=0x7f9950005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.519+0000 7f996d23a640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9950004d00 0x7f9950005170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f996806a5f0 tx=0x7f99500057f0 comp rx=0 tx=0).stop 2023-10-06T13:52:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.519+0000 7f996d23a640 1 -- 172.21.15.19:0/983106925 shutdown_connections 2023-10-06T13:52:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.519+0000 7f996d23a640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9950004d00 0x7f9950005170 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.519+0000 7f996d23a640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9950003970 0x7f9950003d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.519+0000 7f996d23a640 1 --2- 172.21.15.19:0/983106925 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f99540097c0 0x7f99500047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.519+0000 7f996d23a640 1 -- 172.21.15.19:0/983106925 >> 172.21.15.19:0/983106925 conn(0x7f99680b6e00 msgr2=0x7f99680b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.520+0000 7f996d23a640 1 -- 172.21.15.19:0/983106925 shutdown_connections 2023-10-06T13:52:20.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.520+0000 7f996d23a640 1 -- 172.21.15.19:0/983106925 wait complete. 2023-10-06T13:52:20.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.521+0000 7f996d23a640 10 monclient: build_initial_monmap 2023-10-06T13:52:20.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.521+0000 7f996d23a640 10 monclient: monmap: 2023-10-06T13:52:20.527 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:52:20.527 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:20.527 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:52:20.527 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:52:20.527 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:20.527 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:20.527 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:20.527 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:20.527 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:20.527 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:52:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.521+0000 7f996d23a640 1 librados: starting msgr at 2023-10-06T13:52:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.521+0000 7f996d23a640 1 librados: starting objecter 2023-10-06T13:52:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.522+0000 7f996d23a640 1 Processor -- start 2023-10-06T13:52:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.522+0000 7f996d23a640 1 -- start start 2023-10-06T13:52:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.522+0000 7f996d23a640 1 librados: setting wanted keys 2023-10-06T13:52:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.522+0000 7f996d23a640 1 librados: calling monclient init 2023-10-06T13:52:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.522+0000 7f996d23a640 10 monclient: init 2023-10-06T13:52:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 10 monclient: _add_conns ranks=[2,1,0] 2023-10-06T13:52:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9950003970 0x7f99681d8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 10 monclient(hunting): picked mon.noname-c con 0x7f9950003970 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9950004d00 0x7f9968134f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:20.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 10 monclient(hunting): picked mon.noname-b con 0x7f9950004d00 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:20.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99540097c0 0x7f99681354a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:20.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 10 monclient(hunting): picked mon.noname-a con 0x7f99540097c0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:20.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:20.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99681d9350 con 0x7f99540097c0 2023-10-06T13:52:20.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:20.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99681359e0 con 0x7f9950004d00 2023-10-06T13:52:20.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:20.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9968138f00 con 0x7f9950003970 2023-10-06T13:52:20.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:20.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.523+0000 7f996d23a640 10 monclient(hunting): authenticate will time out at 1268.709602s 2023-10-06T13:52:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.537+0000 7f996659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9950004d00 0x7f9968134f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:20.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.538+0000 7f996659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9950004d00 0x7f9968134f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:55802/0 (socket says 172.21.15.19:55802) 2023-10-06T13:52:20.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.538+0000 7f996659c640 1 -- 172.21.15.19:0/559935202 learned_addr learned my addr 172.21.15.19:0/559935202 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:52:20.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.538+0000 7f996659c640 10 monclient(hunting): get_auth_request con 0x7f9950004d00 auth_method 0 2023-10-06T13:52:20.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.538+0000 7f996659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:20.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.538+0000 7f996659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:20.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.538+0000 7f996659c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:20.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.541+0000 7f996659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:20.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.541+0000 7f996659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:20.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.541+0000 7f996659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:20.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.542+0000 7f996759e640 1 --2- 172.21.15.19:0/559935202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99540097c0 0x7f99681354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:20.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.543+0000 7f996759e640 10 monclient(hunting): get_auth_request con 0x7f99540097c0 auth_method 0 2023-10-06T13:52:20.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.543+0000 7f996759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:20.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.543+0000 7f996759e640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:20.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.543+0000 7f996759e640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:20.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.544+0000 7f996759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:20.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.544+0000 7f996759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:20.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.544+0000 7f996759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:20.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.544+0000 7f996659c640 10 monclient(hunting): handle_auth_done global_id 4105 payload 824 2023-10-06T13:52:20.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.544+0000 7f996659c640 1 -- 172.21.15.19:0/559935202 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9950003970 msgr2=0x7f99681d8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:20.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.544+0000 7f996659c640 1 --2- 172.21.15.19:0/559935202 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9950003970 0x7f99681d8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:20.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.544+0000 7f996659c640 1 -- 172.21.15.19:0/559935202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99540097c0 msgr2=0x7f99681354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:20.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.544+0000 7f996659c640 1 --2- 172.21.15.19:0/559935202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99540097c0 0x7f99681354a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:20.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.544+0000 7f996659c640 10 monclient: _finish_hunting 0 2023-10-06T13:52:20.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.544+0000 7f996659c640 1 monclient: found mon.noname-b 2023-10-06T13:52:20.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.544+0000 7f996659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:20.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.544+0000 7f996659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:52:20.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.544+0000 7f996659c640 1 -- 172.21.15.19:0/559935202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9968135f40 con 0x7f9950004d00 2023-10-06T13:52:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.583+0000 7f996659c640 1 --2- 172.21.15.19:0/559935202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9950004d00 0x7f9968134f60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f995c01bec0 tx=0x7f995c005420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.583+0000 7f994f7fe640 10 client.?.objecter ms_handle_connect 0x7f9950004d00 2023-10-06T13:52:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.583+0000 7f994f7fe640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f994f7fe640 1 -- 172.21.15.19:0/559935202 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c019700 con 0x7f9950004d00 2023-10-06T13:52:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f994f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f994f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:52:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f994f7fe640 10 monclient: dump: 2023-10-06T13:52:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:52:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:20.596 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:52:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f994f7fe640 10 monclient: _finish_auth 0 2023-10-06T13:52:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f994f7fe640 10 monclient: _check_auth_tickets 2023-10-06T13:52:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f994f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f994f7fe640 1 -- 172.21.15.19:0/559935202 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c021040 con 0x7f9950004d00 2023-10-06T13:52:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f994f7fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f994f7fe640 1 -- 172.21.15.19:0/559935202 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c019a60 con 0x7f9950004d00 2023-10-06T13:52:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f994f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f994f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f994f7fe640 10 monclient: dump: 2023-10-06T13:52:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:52:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:20.598 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:20.598 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:20.598 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:20.598 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:20.598 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:52:20.598 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f996d23a640 5 monclient: authenticate success, global_id 4105 2023-10-06T13:52:20.598 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f996d23a640 10 monclient: _renew_subs 2023-10-06T13:52:20.598 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f996d23a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:20.598 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f996d23a640 1 -- 172.21.15.19:0/559935202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9968138050 con 0x7f9950004d00 2023-10-06T13:52:20.598 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f996559a640 4 set_mon_vals no callback set 2023-10-06T13:52:20.598 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f996d23a640 10 client.4105.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:20.598 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f996d23a640 10 monclient: _renew_subs 2023-10-06T13:52:20.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f996d23a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:20.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f996d23a640 1 -- 172.21.15.19:0/559935202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99681d8f60 con 0x7f9950004d00 2023-10-06T13:52:20.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f996d23a640 1 librados: init done 2023-10-06T13:52:20.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f996d23a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:20.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f996d23a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:20.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f996d23a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:20.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.594+0000 7f996d23a640 1 -- 172.21.15.19:0/559935202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99681d9180 con 0x7f9950004d00 2023-10-06T13:52:20.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.623+0000 7f994f7fe640 1 -- 172.21.15.19:0/559935202 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f995c019bc0 con 0x7f9950004d00 2023-10-06T13:52:20.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.623+0000 7f994f7fe640 1 -- 172.21.15.19:0/559935202 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f995c038bb0 con 0x7f9950004d00 2023-10-06T13:52:20.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.623+0000 7f994f7fe640 10 client.4105.objecter ms_dispatch 0x7f9968000b90 osd_map(1..1 src has 1..1) v4 2023-10-06T13:52:20.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.623+0000 7f994f7fe640 3 client.4105.objecter handle_osd_map got epochs [1,1] > 0 2023-10-06T13:52:20.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.623+0000 7f994f7fe640 3 client.4105.objecter handle_osd_map decoding full epoch 1 2023-10-06T13:52:20.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.623+0000 7f994f7fe640 20 client.4105.objecter dump_active .. 0 homeless 2023-10-06T13:52:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.793+0000 7f994f7fe640 1 -- 172.21.15.19:0/559935202 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f995c019e70 con 0x7f9950004d00 2023-10-06T13:52:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.793+0000 7f994f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.793+0000 7f994f7fe640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:20.861 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.860+0000 7f996d23a640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2023-10-06T13:52:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.860+0000 7f996d23a640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2023-10-06T13:52:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.860+0000 7f996d23a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:20.860+0000 7f996d23a640 1 -- 172.21.15.19:0/559935202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f99680678e0 con 0x7f9950004d00 2023-10-06T13:52:21.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.113+0000 7f994f7fe640 1 -- 172.21.15.19:0/559935202 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f995c03c090 con 0x7f9950004d00 2023-10-06T13:52:21.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.113+0000 7f994f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2023-10-06T13:52:21.116 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.113+0000 7f994f7fe640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2023-10-06T13:52:21.116 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2023-10-06T13:52:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.114+0000 7f996d23a640 10 librados: watch_flush enter 2023-10-06T13:52:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.114+0000 7f996d23a640 10 librados: watch_flush exit 2023-10-06T13:52:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.114+0000 7f996d23a640 20 client.4105.objecter shutdown clearing up homeless session... 2023-10-06T13:52:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.114+0000 7f996d23a640 10 client.4105.objecter successfully canceled tick 2023-10-06T13:52:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.115+0000 7f996d23a640 10 monclient: shutdown 2023-10-06T13:52:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.115+0000 7f996d23a640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.115+0000 7f996d23a640 1 -- 172.21.15.19:0/559935202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9950004d00 msgr2=0x7f9968134f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:21.118 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.115+0000 7f996d23a640 1 --2- 172.21.15.19:0/559935202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9950004d00 0x7f9968134f60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f995c01bec0 tx=0x7f995c005420 comp rx=0 tx=0).stop 2023-10-06T13:52:21.118 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.115+0000 7f996d23a640 1 -- 172.21.15.19:0/559935202 shutdown_connections 2023-10-06T13:52:21.118 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.115+0000 7f996d23a640 1 --2- 172.21.15.19:0/559935202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99540097c0 0x7f99681354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:21.118 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.115+0000 7f996d23a640 1 --2- 172.21.15.19:0/559935202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9950004d00 0x7f9968134f60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:21.118 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.115+0000 7f996d23a640 1 --2- 172.21.15.19:0/559935202 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9950003970 0x7f99681d8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:21.118 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.115+0000 7f996d23a640 1 -- 172.21.15.19:0/559935202 >> 172.21.15.19:0/559935202 conn(0x7f99680b6e00 msgr2=0x7f99680b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:21.118 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.115+0000 7f996d23a640 1 -- 172.21.15.19:0/559935202 shutdown_connections 2023-10-06T13:52:21.118 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.115+0000 7f996d23a640 1 -- 172.21.15.19:0/559935202 wait complete. 2023-10-06T13:52:21.118 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:21.115+0000 7f996d23a640 1 librados: shutdown 2023-10-06T13:52:21.131 INFO:tasks.ceph:check_enable_crimson: False 2023-10-06T13:52:21.131 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T13:52:21.131 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:52:21.131 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T13:52:21.148 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:21.147+0000 7f6f00a18040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T13:52:21.161 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:52:21.162 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T13:52:21.200 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:21.198+0000 7f6f00a18040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T13:52:21.236 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:52:21.236 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T13:52:21.308 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:52:21.308 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T13:52:21.324 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:21.322+0000 7f4569a0e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T13:52:21.354 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:52:21.354 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T13:52:21.379 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:21.378+0000 7f4569a0e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T13:52:21.381 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:52:21.381 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T13:52:21.447 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:52:21.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T13:52:21.513 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-10-06T13:52:21.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T13:52:21.580 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new b6cb75c5-86fb-48b9-895c-fb1d72bd4746 0 2023-10-06T13:52:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.718+0000 7f297f0c4640 10 monclient: get_monmap_and_config 2023-10-06T13:52:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.718+0000 7f297f0c4640 10 monclient: build_initial_monmap 2023-10-06T13:52:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.718+0000 7f297f0c4640 10 monclient: monmap: 2023-10-06T13:52:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:52:21.718682+0000 2023-10-06T13:52:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:52:21.718682+0000 2023-10-06T13:52:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:52:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:52:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:52:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:52:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:52:21.721 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:21.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.718+0000 7f297f0c4640 1 Processor -- start 2023-10-06T13:52:21.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.718+0000 7f297f0c4640 1 -- start start 2023-10-06T13:52:21.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.718+0000 7f297f0c4640 10 monclient: init 2023-10-06T13:52:21.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.719+0000 7f297f0c4640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:21.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.719+0000 7f297f0c4640 10 monclient: _add_conns ranks=[4,2,0,1,3] 2023-10-06T13:52:21.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.719+0000 7f297f0c4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f29780a8bb0 0x7f29780b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:21.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.719+0000 7f297f0c4640 10 monclient(hunting): picked mon.noname-c con 0x7f29780a8bb0 addr v2:172.21.15.83:3300/0 2023-10-06T13:52:21.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.719+0000 7f297f0c4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f29780b0da0 0x7f2978140b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:21.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.719+0000 7f297f0c4640 10 monclient(hunting): picked mon.noname-b con 0x7f29780b0da0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:21.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.719+0000 7f297f0c4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2978141090 addr v1:172.21.15.19:6789/0 2023-10-06T13:52:21.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.719+0000 7f297f0c4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29780aa800 con 0x7f2978141090 2023-10-06T13:52:21.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.719+0000 7f297f0c4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:21.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.719+0000 7f297f0c4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29780a9360 con 0x7f29780a8bb0 2023-10-06T13:52:21.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.719+0000 7f297f0c4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.719+0000 7f297f0c4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29780b11f0 con 0x7f29780b0da0 2023-10-06T13:52:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.719+0000 7f297f0c4640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.719+0000 7f297f0c4640 10 monclient(hunting): authenticate will time out at 1270.051749s 2023-10-06T13:52:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.720+0000 7f297ce60640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f29780a8bb0 0x7f29780b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.720+0000 7f297ce60640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f29780a8bb0 0x7f29780b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48284/0 (socket says 172.21.15.83:48284) 2023-10-06T13:52:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.720+0000 7f297ce60640 1 -- 172.21.15.83:0/1362073794 learned_addr learned my addr 172.21.15.83:0/1362073794 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-10-06T13:52:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.720+0000 7f297ce60640 10 monclient(hunting): get_auth_request con 0x7f29780a8bb0 auth_method 0 2023-10-06T13:52:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.720+0000 7f297ce60640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.720+0000 7f297ce60640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.720+0000 7f297ce60640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.720+0000 7f296ffff640 1 --2- 172.21.15.83:0/1362073794 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f29780b0da0 0x7f2978140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.721+0000 7f297ce60640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.721+0000 7f297ce60640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.721+0000 7f297ce60640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.721+0000 7f296ffff640 10 monclient(hunting): get_auth_request con 0x7f29780b0da0 auth_method 0 2023-10-06T13:52:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.721+0000 7f296ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.721+0000 7f296ffff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.721+0000 7f296ffff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.722+0000 7f296ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.722+0000 7f296ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.722+0000 7f296ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.722+0000 7f297ce60640 10 monclient(hunting): handle_auth_done global_id 4108 payload 274 2023-10-06T13:52:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.722+0000 7f297ce60640 1 -- 172.21.15.83:0/1362073794 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f29780b0da0 msgr2=0x7f2978140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.722+0000 7f297ce60640 1 --2- 172.21.15.83:0/1362073794 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f29780b0da0 0x7f2978140b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.722+0000 7f297ce60640 1 -- 172.21.15.83:0/1362073794 >> v1:172.21.15.19:6789/0 conn(0x7f2978141090 legacy=0x7f29780ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.722+0000 7f297ce60640 10 monclient: _finish_hunting 0 2023-10-06T13:52:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.722+0000 7f297ce60640 1 monclient: found mon.noname-c 2023-10-06T13:52:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.722+0000 7f297ce60640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.722+0000 7f297ce60640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:52:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.722+0000 7f297ce60640 1 -- 172.21.15.83:0/1362073794 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29780aaa50 con 0x7f29780a8bb0 2023-10-06T13:52:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.724+0000 7f297ce60640 1 --2- 172.21.15.83:0/1362073794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29780a8bb0 0x7f29780b0860 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2974009870 tx=0x7f297402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bf9e2c61548963c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f296f7fe640 1 -- 172.21.15.83:0/1362073794 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974031640 con 0x7f29780a8bb0 2023-10-06T13:52:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f296f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f296f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:52:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f296f7fe640 10 monclient: dump: 2023-10-06T13:52:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:21.731 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f296f7fe640 10 monclient: _finish_auth 0 2023-10-06T13:52:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f296f7fe640 10 monclient: _check_auth_tickets 2023-10-06T13:52:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f296f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f296f7fe640 1 -- 172.21.15.83:0/1362073794 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29740317a0 con 0x7f29780a8bb0 2023-10-06T13:52:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f297f0c4640 5 monclient: authenticate success, global_id 4108 2023-10-06T13:52:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f297f0c4640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:52:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f296f7fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f297f0c4640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f296f7fe640 1 -- 172.21.15.83:0/1362073794 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974031b30 con 0x7f29780a8bb0 2023-10-06T13:52:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f296f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f297f0c4640 4 set_mon_vals no callback set 2023-10-06T13:52:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f296f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.728+0000 7f296f7fe640 10 monclient: dump: 2023-10-06T13:52:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:21.733 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.729+0000 7f297f0c4640 10 monclient: shutdown 2023-10-06T13:52:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.729+0000 7f297f0c4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.729+0000 7f297f0c4640 1 -- 172.21.15.83:0/1362073794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29780a8bb0 msgr2=0x7f29780b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.729+0000 7f297f0c4640 1 --2- 172.21.15.83:0/1362073794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29780a8bb0 0x7f29780b0860 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2974009870 tx=0x7f297402b8f0 comp rx=0 tx=0).stop 2023-10-06T13:52:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.729+0000 7f297f0c4640 1 -- 172.21.15.83:0/1362073794 shutdown_connections 2023-10-06T13:52:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.729+0000 7f297f0c4640 1 --2- 172.21.15.83:0/1362073794 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f29780b0da0 0x7f2978140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.729+0000 7f297f0c4640 1 --2- 172.21.15.83:0/1362073794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29780a8bb0 0x7f29780b0860 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.729+0000 7f297f0c4640 1 -- 172.21.15.83:0/1362073794 >> 172.21.15.83:0/1362073794 conn(0x7f2978065f50 msgr2=0x7f29780b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.729+0000 7f297f0c4640 1 -- 172.21.15.83:0/1362073794 shutdown_connections 2023-10-06T13:52:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.729+0000 7f297f0c4640 1 -- 172.21.15.83:0/1362073794 wait complete. 2023-10-06T13:52:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.729+0000 7f297f0c4640 10 monclient: build_initial_monmap 2023-10-06T13:52:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.729+0000 7f297f0c4640 10 monclient: monmap: 2023-10-06T13:52:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 0.000000 2023-10-06T13:52:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:created 0.000000 2023-10-06T13:52:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:21.737 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 1 librados: starting msgr at 2023-10-06T13:52:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 1 librados: starting objecter 2023-10-06T13:52:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 1 Processor -- start 2023-10-06T13:52:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 1 -- start start 2023-10-06T13:52:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 1 librados: setting wanted keys 2023-10-06T13:52:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 1 librados: calling monclient init 2023-10-06T13:52:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 10 monclient: init 2023-10-06T13:52:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T13:52:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29780a8bb0 0x7f29781c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 10 monclient(hunting): picked mon.noname-a con 0x7f29780a8bb0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29780b0da0 0x7f29781d4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 10 monclient(hunting): picked mon.noname-b con 0x7f29780b0da0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2978141090 0x7f297806f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 10 monclient(hunting): picked mon.noname-c con 0x7f2978141090 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297806eaa0 con 0x7f29780a8bb0 2023-10-06T13:52:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29781d57f0 con 0x7f29780b0da0 2023-10-06T13:52:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29781c8ef0 con 0x7f2978141090 2023-10-06T13:52:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.730+0000 7f297f0c4640 10 monclient(hunting): authenticate will time out at 1270.063176s 2023-10-06T13:52:21.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.755+0000 7f297d661640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2978141090 0x7f297806f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.756+0000 7f297d661640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2978141090 0x7f297806f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.83:57620/0 (socket says 172.21.15.83:57620) 2023-10-06T13:52:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.756+0000 7f297d661640 1 -- 172.21.15.83:0/2508976920 learned_addr learned my addr 172.21.15.83:0/2508976920 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-10-06T13:52:21.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.756+0000 7f297d661640 10 monclient(hunting): get_auth_request con 0x7f2978141090 auth_method 0 2023-10-06T13:52:21.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.756+0000 7f297d661640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:21.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.756+0000 7f297d661640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:21.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.756+0000 7f297d661640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:21.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.756+0000 7f297ce60640 1 --2- 172.21.15.83:0/2508976920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29780a8bb0 0x7f29781c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.757+0000 7f297ce60640 10 monclient(hunting): get_auth_request con 0x7f29780a8bb0 auth_method 0 2023-10-06T13:52:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.757+0000 7f297ce60640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.757+0000 7f297ce60640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.757+0000 7f297ce60640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.757+0000 7f297ce60640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.757+0000 7f297ce60640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.757+0000 7f297ce60640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.757+0000 7f297d661640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.757+0000 7f297d661640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.757+0000 7f297d661640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.759+0000 7f296ffff640 1 --2- 172.21.15.83:0/2508976920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29780b0da0 0x7f29781d4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.760+0000 7f297ce60640 10 monclient(hunting): handle_auth_done global_id 4116 payload 824 2023-10-06T13:52:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.760+0000 7f297ce60640 1 -- 172.21.15.83:0/2508976920 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2978141090 msgr2=0x7f297806f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.760+0000 7f297ce60640 1 --2- 172.21.15.83:0/2508976920 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2978141090 0x7f297806f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.760+0000 7f297ce60640 1 -- 172.21.15.83:0/2508976920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29780b0da0 msgr2=0x7f29781d4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.760+0000 7f297ce60640 1 --2- 172.21.15.83:0/2508976920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29780b0da0 0x7f29781d4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.760+0000 7f297ce60640 10 monclient: _finish_hunting 0 2023-10-06T13:52:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.760+0000 7f297ce60640 1 monclient: found mon.noname-a 2023-10-06T13:52:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.760+0000 7f297ce60640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.760+0000 7f297ce60640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:52:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.760+0000 7f297ce60640 1 -- 172.21.15.83:0/2508976920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29740094d0 con 0x7f29780a8bb0 2023-10-06T13:52:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.763+0000 7f297ce60640 1 --2- 172.21.15.83:0/2508976920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29780a8bb0 0x7f29781c8350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2974033000 tx=0x7f29740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.763+0000 7f296d7fa640 10 client.?.objecter ms_handle_connect 0x7f29780a8bb0 2023-10-06T13:52:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.763+0000 7f296d7fa640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:21.768 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:21.766+0000 7f6f00a18040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T13:52:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f296d7fa640 1 -- 172.21.15.83:0/2508976920 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297403ac60 con 0x7f29780a8bb0 2023-10-06T13:52:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f296d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f296d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:52:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f296d7fa640 10 monclient: dump: 2023-10-06T13:52:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:21.773 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f296d7fa640 10 monclient: _finish_auth 0 2023-10-06T13:52:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f296d7fa640 10 monclient: _check_auth_tickets 2023-10-06T13:52:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f296d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f296d7fa640 1 -- 172.21.15.83:0/2508976920 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297403ade0 con 0x7f29780a8bb0 2023-10-06T13:52:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f296d7fa640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f296d7fa640 1 -- 172.21.15.83:0/2508976920 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974044630 con 0x7f29780a8bb0 2023-10-06T13:52:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f296d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f296d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:52:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f296d7fa640 10 monclient: dump: 2023-10-06T13:52:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:21.774 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f297f0c4640 5 monclient: authenticate success, global_id 4116 2023-10-06T13:52:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f297f0c4640 10 monclient: _renew_subs 2023-10-06T13:52:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f297f0c4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:52:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f296effd640 4 set_mon_vals no callback set 2023-10-06T13:52:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.769+0000 7f297f0c4640 1 -- 172.21.15.83:0/2508976920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297806f550 con 0x7f29780a8bb0 2023-10-06T13:52:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.770+0000 7f297f0c4640 10 client.4116.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.770+0000 7f297f0c4640 10 monclient: _renew_subs 2023-10-06T13:52:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.770+0000 7f297f0c4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:52:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.770+0000 7f297f0c4640 1 -- 172.21.15.83:0/2508976920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297806e7a0 con 0x7f29780a8bb0 2023-10-06T13:52:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.770+0000 7f297f0c4640 1 librados: init done 2023-10-06T13:52:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.770+0000 7f297f0c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.770+0000 7f297f0c4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.770+0000 7f297f0c4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:52:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.770+0000 7f297f0c4640 1 -- 172.21.15.83:0/2508976920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29781d4dc0 con 0x7f29780a8bb0 2023-10-06T13:52:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.774+0000 7f296d7fa640 1 -- 172.21.15.83:0/2508976920 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f297402f050 con 0x7f29780a8bb0 2023-10-06T13:52:21.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.774+0000 7f296d7fa640 1 -- 172.21.15.83:0/2508976920 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f297404d490 con 0x7f29780a8bb0 2023-10-06T13:52:21.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.774+0000 7f296d7fa640 10 client.4116.objecter ms_dispatch 0x7f2978000b90 osd_map(2..2 src has 1..2) v4 2023-10-06T13:52:21.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.774+0000 7f296d7fa640 3 client.4116.objecter handle_osd_map got epochs [2,2] > 0 2023-10-06T13:52:21.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.774+0000 7f296d7fa640 3 client.4116.objecter handle_osd_map decoding full epoch 2 2023-10-06T13:52:21.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.774+0000 7f296d7fa640 20 client.4116.objecter dump_active .. 0 homeless 2023-10-06T13:52:21.820 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:21.819+0000 7f6f00a18040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T13:52:21.865 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:21.863+0000 7f6f00a18040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T13:52:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.895+0000 7f296d7fa640 1 -- 172.21.15.83:0/2508976920 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f2974035020 con 0x7f29780a8bb0 2023-10-06T13:52:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.895+0000 7f296d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.895+0000 7f296d7fa640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:21.919 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:21.918+0000 7f6f00a18040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T13:52:21.947 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:21.945+0000 7f4569a0e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T13:52:21.963 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:21.961+0000 7f6f00a18040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T13:52:21.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.967+0000 7f297f0c4640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "b6cb75c5-86fb-48b9-895c-fb1d72bd4746", "id": 0}] 2023-10-06T13:52:21.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.967+0000 7f297f0c4640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "b6cb75c5-86fb-48b9-895c-fb1d72bd4746", "id": 0}] 2023-10-06T13:52:21.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.967+0000 7f297f0c4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:52:21.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:21.967+0000 7f297f0c4640 1 -- 172.21.15.83:0/2508976920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b6cb75c5-86fb-48b9-895c-fb1d72bd4746", "id": 0} v 0) v1 -- 0x7f297806ed30 con 0x7f29780a8bb0 2023-10-06T13:52:22.002 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:22.000+0000 7f4569a0e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T13:52:22.049 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:22.047+0000 7f4569a0e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T13:52:22.066 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:22.065+0000 7f6f00a18040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T13:52:22.107 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:22.105+0000 7f4569a0e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T13:52:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.121+0000 7f296d7fa640 1 -- 172.21.15.83:0/2508976920 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b6cb75c5-86fb-48b9-895c-fb1d72bd4746", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f297402d040 con 0x7f29780a8bb0 2023-10-06T13:52:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.121+0000 7f296d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "b6cb75c5-86fb-48b9-895c-fb1d72bd4746", "id": 0}] 2023-10-06T13:52:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.121+0000 7f296d7fa640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:52:22.123 INFO:teuthology.orchestra.run.smithi083.stdout:0 2023-10-06T13:52:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.122+0000 7f297f0c4640 10 librados: watch_flush enter 2023-10-06T13:52:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.122+0000 7f297f0c4640 10 librados: watch_flush exit 2023-10-06T13:52:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.122+0000 7f297f0c4640 20 client.4116.objecter shutdown clearing up homeless session... 2023-10-06T13:52:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.122+0000 7f297f0c4640 10 client.4116.objecter successfully canceled tick 2023-10-06T13:52:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.122+0000 7f297f0c4640 10 monclient: shutdown 2023-10-06T13:52:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.122+0000 7f297f0c4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.122+0000 7f297f0c4640 1 -- 172.21.15.83:0/2508976920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29780a8bb0 msgr2=0x7f29781c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.122+0000 7f297f0c4640 1 --2- 172.21.15.83:0/2508976920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29780a8bb0 0x7f29781c8350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2974033000 tx=0x7f29740046c0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.122+0000 7f297f0c4640 1 -- 172.21.15.83:0/2508976920 shutdown_connections 2023-10-06T13:52:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.122+0000 7f297f0c4640 1 --2- 172.21.15.83:0/2508976920 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2978141090 0x7f297806f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.122+0000 7f297f0c4640 1 --2- 172.21.15.83:0/2508976920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29780b0da0 0x7f29781d4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.122+0000 7f297f0c4640 1 --2- 172.21.15.83:0/2508976920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29780a8bb0 0x7f29781c8350 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.122+0000 7f297f0c4640 1 -- 172.21.15.83:0/2508976920 >> 172.21.15.83:0/2508976920 conn(0x7f2978065f50 msgr2=0x7f29780af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.123+0000 7f297f0c4640 1 -- 172.21.15.83:0/2508976920 shutdown_connections 2023-10-06T13:52:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.123+0000 7f297f0c4640 1 -- 172.21.15.83:0/2508976920 wait complete. 2023-10-06T13:52:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.123+0000 7f297f0c4640 1 librados: shutdown 2023-10-06T13:52:22.134 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new ba828cb4-a4f6-4fea-a02e-24d9cf597ea7 1 2023-10-06T13:52:22.152 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:22.150+0000 7f4569a0e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T13:52:22.323 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:22.262+0000 7f4569a0e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T13:52:22.323 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:22.166+0000 7f6f00a18040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T13:52:22.323 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:22.215+0000 7f6f00a18040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T13:52:22.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.233+0000 7f2e28d77640 10 monclient: get_monmap_and_config 2023-10-06T13:52:22.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.233+0000 7f2e28d77640 10 monclient: build_initial_monmap 2023-10-06T13:52:22.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.233+0000 7f2e28d77640 10 monclient: monmap: 2023-10-06T13:52:22.331 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:22.331 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:22.331 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:52:22.234022+0000 2023-10-06T13:52:22.331 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:52:22.234022+0000 2023-10-06T13:52:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:52:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:52:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:52:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:52:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:52:22.332 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 1 Processor -- start 2023-10-06T13:52:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 1 -- start start 2023-10-06T13:52:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 10 monclient: init 2023-10-06T13:52:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 10 monclient: _add_conns ranks=[2,3,4,0,1] 2023-10-06T13:52:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2e240b3990 0x7f2e240af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 10 monclient(hunting): picked mon.noname-b con 0x7f2e240b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2e240b3d60 0x7f2e24141500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 10 monclient(hunting): picked mon.noname-a con 0x7f2e240b3d60 addr v2:172.21.15.19:3300/0 2023-10-06T13:52:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e24141a40 0x7f2e240b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 10 monclient(hunting): picked mon.noname-c con 0x7f2e24141a40 addr v2:172.21.15.83:3300/0 2023-10-06T13:52:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e24140dd0 con 0x7f2e240b3d60 2023-10-06T13:52:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e240aebe0 con 0x7f2e24141a40 2023-10-06T13:52:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e24141070 con 0x7f2e240b3990 2023-10-06T13:52:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.234+0000 7f2e28d77640 10 monclient(hunting): authenticate will time out at 1270.567083s 2023-10-06T13:52:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.235+0000 7f2e22d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e24141a40 0x7f2e240b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.235+0000 7f2e21d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2e240b3d60 0x7f2e24141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.235+0000 7f2e2259c640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2e240b3990 0x7f2e240af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.235+0000 7f2e22d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e24141a40 0x7f2e240b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48316/0 (socket says 172.21.15.83:48316) 2023-10-06T13:52:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.235+0000 7f2e22d9d640 1 -- 172.21.15.83:0/950323078 learned_addr learned my addr 172.21.15.83:0/950323078 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-10-06T13:52:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.235+0000 7f2e22d9d640 10 monclient(hunting): get_auth_request con 0x7f2e24141a40 auth_method 0 2023-10-06T13:52:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.235+0000 7f2e22d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.235+0000 7f2e22d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.235+0000 7f2e22d9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.235+0000 7f2e21d9b640 10 monclient(hunting): get_auth_request con 0x7f2e240b3d60 auth_method 0 2023-10-06T13:52:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.235+0000 7f2e21d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.235+0000 7f2e21d9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.235+0000 7f2e21d9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.236+0000 7f2e2259c640 10 monclient(hunting): get_auth_request con 0x7f2e240b3990 auth_method 0 2023-10-06T13:52:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.236+0000 7f2e2259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.236+0000 7f2e2259c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.236+0000 7f2e2259c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.236+0000 7f2e22d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.236+0000 7f2e22d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.236+0000 7f2e22d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.236+0000 7f2e21d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.236+0000 7f2e21d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.236+0000 7f2e21d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.236+0000 7f2e2259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.236+0000 7f2e2259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.236+0000 7f2e2259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.237+0000 7f2e22d9d640 10 monclient(hunting): handle_auth_done global_id 4111 payload 274 2023-10-06T13:52:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.237+0000 7f2e22d9d640 1 -- 172.21.15.83:0/950323078 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2e240b3990 msgr2=0x7f2e240af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.237+0000 7f2e22d9d640 1 --2- 172.21.15.83:0/950323078 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2e240b3990 0x7f2e240af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.237+0000 7f2e22d9d640 1 -- 172.21.15.83:0/950323078 >> v2:172.21.15.19:3300/0 conn(0x7f2e240b3d60 msgr2=0x7f2e24141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.237+0000 7f2e22d9d640 1 --2- 172.21.15.83:0/950323078 >> v2:172.21.15.19:3300/0 conn(0x7f2e240b3d60 0x7f2e24141500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.237+0000 7f2e22d9d640 10 monclient: _finish_hunting 0 2023-10-06T13:52:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.237+0000 7f2e22d9d640 1 monclient: found mon.noname-c 2023-10-06T13:52:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.237+0000 7f2e22d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:22.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.237+0000 7f2e22d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:52:22.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.237+0000 7f2e22d9d640 1 -- 172.21.15.83:0/950323078 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e240aee30 con 0x7f2e24141a40 2023-10-06T13:52:22.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.239+0000 7f2e22d9d640 1 --2- 172.21.15.83:0/950323078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e24141a40 0x7f2e240b00c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2e240b1080 tx=0x7f2e1c02be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9727bba50683d1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:22.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e2159a640 1 -- 172.21.15.83:0/950323078 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c02c9e0 con 0x7f2e24141a40 2023-10-06T13:52:22.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e2159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:22.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e2159a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:52:22.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e2159a640 10 monclient: dump: 2023-10-06T13:52:22.337 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:22.337 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:22.337 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.337 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.337 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:22.338 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:22.338 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:22.338 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:22.338 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:22.338 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:22.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e2159a640 10 monclient: _finish_auth 0 2023-10-06T13:52:22.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e2159a640 10 monclient: _check_auth_tickets 2023-10-06T13:52:22.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e2159a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:22.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e2159a640 1 -- 172.21.15.83:0/950323078 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c02cb40 con 0x7f2e24141a40 2023-10-06T13:52:22.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e28d77640 5 monclient: authenticate success, global_id 4111 2023-10-06T13:52:22.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e28d77640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:52:22.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e2159a640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:22.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e2159a640 1 -- 172.21.15.83:0/950323078 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0343f0 con 0x7f2e24141a40 2023-10-06T13:52:22.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e28d77640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:22.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e28d77640 4 set_mon_vals no callback set 2023-10-06T13:52:22.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e2159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:22.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e2159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:22.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e2159a640 10 monclient: dump: 2023-10-06T13:52:22.339 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:22.339 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:22.339 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.339 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.339 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:22.339 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:22.339 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:22.340 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:22.340 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:22.340 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:22.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e28d77640 10 monclient: shutdown 2023-10-06T13:52:22.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e28d77640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:22.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e28d77640 1 -- 172.21.15.83:0/950323078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e24141a40 msgr2=0x7f2e240b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:22.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.244+0000 7f2e28d77640 1 --2- 172.21.15.83:0/950323078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e24141a40 0x7f2e240b00c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2e240b1080 tx=0x7f2e1c02be40 comp rx=0 tx=0).stop 2023-10-06T13:52:22.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.245+0000 7f2e28d77640 1 -- 172.21.15.83:0/950323078 shutdown_connections 2023-10-06T13:52:22.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.245+0000 7f2e28d77640 1 --2- 172.21.15.83:0/950323078 >> v2:172.21.15.19:3300/0 conn(0x7f2e240b3d60 0x7f2e24141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.245+0000 7f2e28d77640 1 --2- 172.21.15.83:0/950323078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e24141a40 0x7f2e240b00c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.245+0000 7f2e28d77640 1 --2- 172.21.15.83:0/950323078 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2e240b3990 0x7f2e240af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.245+0000 7f2e28d77640 1 -- 172.21.15.83:0/950323078 >> 172.21.15.83:0/950323078 conn(0x7f2e240b6e00 msgr2=0x7f2e240b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.245+0000 7f2e28d77640 1 -- 172.21.15.83:0/950323078 shutdown_connections 2023-10-06T13:52:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.245+0000 7f2e28d77640 1 -- 172.21.15.83:0/950323078 wait complete. 2023-10-06T13:52:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.245+0000 7f2e28d77640 10 monclient: build_initial_monmap 2023-10-06T13:52:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.245+0000 7f2e28d77640 10 monclient: monmap: 2023-10-06T13:52:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 0.000000 2023-10-06T13:52:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:created 0.000000 2023-10-06T13:52:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:22.342 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:22.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 1 librados: starting msgr at 2023-10-06T13:52:22.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 1 librados: starting objecter 2023-10-06T13:52:22.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 1 Processor -- start 2023-10-06T13:52:22.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 1 -- start start 2023-10-06T13:52:22.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 1 librados: setting wanted keys 2023-10-06T13:52:22.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 1 librados: calling monclient init 2023-10-06T13:52:22.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 10 monclient: init 2023-10-06T13:52:22.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:22.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T13:52:22.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e240b3990 0x7f2e24144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:22.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 10 monclient(hunting): picked mon.noname-b con 0x7f2e240b3990 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e240b3d60 0x7f2e241d9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 10 monclient(hunting): picked mon.noname-a con 0x7f2e240b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2e24141a40 0x7f2e240a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 10 monclient(hunting): picked mon.noname-c con 0x7f2e24141a40 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e241d98e0 con 0x7f2e240b3d60 2023-10-06T13:52:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e241d95c0 con 0x7f2e240b3990 2023-10-06T13:52:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e24144bc0 con 0x7f2e24141a40 2023-10-06T13:52:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.246+0000 7f2e28d77640 10 monclient(hunting): authenticate will time out at 1270.579201s 2023-10-06T13:52:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.247+0000 7f2e2259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e240b3990 0x7f2e24144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.247+0000 7f2e22d9d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2e24141a40 0x7f2e240a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.247+0000 7f2e21d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e240b3d60 0x7f2e241d9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.247+0000 7f2e2259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e240b3990 0x7f2e24144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48324/0 (socket says 172.21.15.83:48324) 2023-10-06T13:52:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.247+0000 7f2e2259c640 1 -- 172.21.15.83:0/1542936916 learned_addr learned my addr 172.21.15.83:0/1542936916 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-10-06T13:52:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.247+0000 7f2e2259c640 10 monclient(hunting): get_auth_request con 0x7f2e240b3990 auth_method 0 2023-10-06T13:52:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.247+0000 7f2e2259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.247+0000 7f2e2259c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.247+0000 7f2e2259c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.247+0000 7f2e21d9b640 10 monclient(hunting): get_auth_request con 0x7f2e240b3d60 auth_method 0 2023-10-06T13:52:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.247+0000 7f2e21d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.247+0000 7f2e21d9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.247+0000 7f2e21d9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.248+0000 7f2e22d9d640 10 monclient(hunting): get_auth_request con 0x7f2e24141a40 auth_method 0 2023-10-06T13:52:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.248+0000 7f2e22d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.248+0000 7f2e22d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.248+0000 7f2e22d9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.248+0000 7f2e2259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.248+0000 7f2e2259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.248+0000 7f2e2259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.248+0000 7f2e21d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.248+0000 7f2e21d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.248+0000 7f2e21d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.249+0000 7f2e22d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.249+0000 7f2e22d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.249+0000 7f2e22d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.250+0000 7f2e2259c640 10 monclient(hunting): handle_auth_done global_id 4114 payload 824 2023-10-06T13:52:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.250+0000 7f2e2259c640 1 -- 172.21.15.83:0/1542936916 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2e24141a40 msgr2=0x7f2e240a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.250+0000 7f2e2259c640 1 --2- 172.21.15.83:0/1542936916 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2e24141a40 0x7f2e240a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.250+0000 7f2e2259c640 1 -- 172.21.15.83:0/1542936916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e240b3d60 msgr2=0x7f2e241d9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.250+0000 7f2e2259c640 1 --2- 172.21.15.83:0/1542936916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e240b3d60 0x7f2e241d9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.250+0000 7f2e2259c640 10 monclient: _finish_hunting 0 2023-10-06T13:52:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.250+0000 7f2e2259c640 1 monclient: found mon.noname-b 2023-10-06T13:52:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.250+0000 7f2e2259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.251+0000 7f2e2259c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:52:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.251+0000 7f2e2259c640 1 -- 172.21.15.83:0/1542936916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c009b30 con 0x7f2e240b3990 2023-10-06T13:52:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.253+0000 7f2e2259c640 1 --2- 172.21.15.83:0/1542936916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e240b3990 0x7f2e24144000 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800e890 tx=0x7f2e1800ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.253+0000 7f2e12ffd640 10 client.?.objecter ms_handle_connect 0x7f2e240b3990 2023-10-06T13:52:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.253+0000 7f2e12ffd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e12ffd640 1 -- 172.21.15.83:0/1542936916 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1800c940 con 0x7f2e240b3990 2023-10-06T13:52:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e12ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e12ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:52:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e12ffd640 10 monclient: dump: 2023-10-06T13:52:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:22.348 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e12ffd640 10 monclient: _finish_auth 0 2023-10-06T13:52:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e12ffd640 10 monclient: _check_auth_tickets 2023-10-06T13:52:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e12ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e12ffd640 1 -- 172.21.15.83:0/1542936916 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1800caa0 con 0x7f2e240b3990 2023-10-06T13:52:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e28d77640 5 monclient: authenticate success, global_id 4114 2023-10-06T13:52:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e28d77640 10 monclient: _renew_subs 2023-10-06T13:52:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e28d77640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e28d77640 1 -- 172.21.15.83:0/1542936916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e241d9e00 con 0x7f2e240b3990 2023-10-06T13:52:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e12ffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e12ffd640 1 -- 172.21.15.83:0/1542936916 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1800ce30 con 0x7f2e240b3990 2023-10-06T13:52:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e12ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e12ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e12ffd640 10 monclient: dump: 2023-10-06T13:52:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:22.350 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e20d99640 4 set_mon_vals no callback set 2023-10-06T13:52:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e28d77640 10 client.4114.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e28d77640 10 monclient: _renew_subs 2023-10-06T13:52:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e28d77640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e28d77640 1 -- 172.21.15.83:0/1542936916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e240a8b70 con 0x7f2e240b3990 2023-10-06T13:52:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.257+0000 7f2e28d77640 1 librados: init done 2023-10-06T13:52:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.258+0000 7f2e28d77640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.258+0000 7f2e28d77640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.258+0000 7f2e28d77640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.258+0000 7f2e28d77640 1 -- 172.21.15.83:0/1542936916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e241d9730 con 0x7f2e240b3990 2023-10-06T13:52:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.260+0000 7f2e12ffd640 1 -- 172.21.15.83:0/1542936916 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2e18005c90 con 0x7f2e240b3990 2023-10-06T13:52:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.260+0000 7f2e12ffd640 1 -- 172.21.15.83:0/1542936916 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f2e18006830 con 0x7f2e240b3990 2023-10-06T13:52:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.260+0000 7f2e12ffd640 10 client.4114.objecter ms_dispatch 0x7f2e24000b90 osd_map(3..3 src has 1..3) v4 2023-10-06T13:52:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.260+0000 7f2e12ffd640 3 client.4114.objecter handle_osd_map got epochs [3,3] > 0 2023-10-06T13:52:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.260+0000 7f2e12ffd640 3 client.4114.objecter handle_osd_map decoding full epoch 3 2023-10-06T13:52:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.260+0000 7f2e12ffd640 20 client.4114.objecter dump_active .. 0 homeless 2023-10-06T13:52:22.351 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:22.274+0000 7f6f00a18040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T13:52:22.351 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:22.322+0000 7f6f00a18040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T13:52:22.352 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:22.335+0000 7f4569a0e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T13:52:22.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.362+0000 7f2e12ffd640 1 -- 172.21.15.83:0/1542936916 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f2e18021060 con 0x7f2e240b3990 2023-10-06T13:52:22.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.362+0000 7f2e12ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:22.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.362+0000 7f2e12ffd640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:22.385 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:22.383+0000 7f4569a0e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T13:52:22.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.435+0000 7f2e28d77640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "ba828cb4-a4f6-4fea-a02e-24d9cf597ea7", "id": 1}] 2023-10-06T13:52:22.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.435+0000 7f2e28d77640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "ba828cb4-a4f6-4fea-a02e-24d9cf597ea7", "id": 1}] 2023-10-06T13:52:22.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.435+0000 7f2e28d77640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:22.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.435+0000 7f2e28d77640 1 -- 172.21.15.83:0/1542936916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ba828cb4-a4f6-4fea-a02e-24d9cf597ea7", "id": 1} v 0) v1 -- 0x7f2e240abdc0 con 0x7f2e240b3990 2023-10-06T13:52:22.438 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:22.437+0000 7f4569a0e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T13:52:22.488 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:22.486+0000 7f4569a0e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T13:52:22.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.514+0000 7f2e12ffd640 1 -- 172.21.15.83:0/1542936916 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ba828cb4-a4f6-4fea-a02e-24d9cf597ea7", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2e18017070 con 0x7f2e240b3990 2023-10-06T13:52:22.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.514+0000 7f2e12ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "ba828cb4-a4f6-4fea-a02e-24d9cf597ea7", "id": 1}] 2023-10-06T13:52:22.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.514+0000 7f2e12ffd640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:52:22.517 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-10-06T13:52:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 10 librados: watch_flush enter 2023-10-06T13:52:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 10 librados: watch_flush exit 2023-10-06T13:52:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 20 client.4114.objecter shutdown clearing up homeless session... 2023-10-06T13:52:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 10 client.4114.objecter successfully canceled tick 2023-10-06T13:52:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 10 monclient: shutdown 2023-10-06T13:52:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 1 -- 172.21.15.83:0/1542936916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e240b3990 msgr2=0x7f2e24144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 1 --2- 172.21.15.83:0/1542936916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e240b3990 0x7f2e24144000 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800e890 tx=0x7f2e1800ed50 comp rx=0 tx=0).stop 2023-10-06T13:52:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 1 -- 172.21.15.83:0/1542936916 shutdown_connections 2023-10-06T13:52:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 1 --2- 172.21.15.83:0/1542936916 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2e24141a40 0x7f2e240a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 1 --2- 172.21.15.83:0/1542936916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e240b3d60 0x7f2e241d9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 1 --2- 172.21.15.83:0/1542936916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e240b3990 0x7f2e24144000 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 1 -- 172.21.15.83:0/1542936916 >> 172.21.15.83:0/1542936916 conn(0x7f2e240b6e00 msgr2=0x7f2e240b4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 1 -- 172.21.15.83:0/1542936916 shutdown_connections 2023-10-06T13:52:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 1 -- 172.21.15.83:0/1542936916 wait complete. 2023-10-06T13:52:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.515+0000 7f2e28d77640 1 librados: shutdown 2023-10-06T13:52:22.527 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new 06bcfab5-d2f1-4447-ba44-143398707ef4 2 2023-10-06T13:52:22.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.665+0000 7f0e474e5640 10 monclient: get_monmap_and_config 2023-10-06T13:52:22.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.665+0000 7f0e474e5640 10 monclient: build_initial_monmap 2023-10-06T13:52:22.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.665+0000 7f0e474e5640 10 monclient: monmap: 2023-10-06T13:52:22.667 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:22.667 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:22.667 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:52:22.666036+0000 2023-10-06T13:52:22.667 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:52:22.666036+0000 2023-10-06T13:52:22.668 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:22.668 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:22.668 INFO:teuthology.orchestra.run.smithi083.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:52:22.668 INFO:teuthology.orchestra.run.smithi083.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:52:22.668 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:52:22.668 INFO:teuthology.orchestra.run.smithi083.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:52:22.668 INFO:teuthology.orchestra.run.smithi083.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:52:22.668 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:22.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.665+0000 7f0e474e5640 1 Processor -- start 2023-10-06T13:52:22.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 1 -- start start 2023-10-06T13:52:22.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 10 monclient: init 2023-10-06T13:52:22.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:22.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2023-10-06T13:52:22.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0e400b3990 0x7f0e400af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 10 monclient(hunting): picked mon.noname-b con 0x7f0e400b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0e400b3d60 0x7f0e40141500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 10 monclient(hunting): picked mon.noname-a con 0x7f0e400b3d60 addr v2:172.21.15.19:3300/0 2023-10-06T13:52:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e40141a40 0x7f0e400b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 10 monclient(hunting): picked mon.noname-c con 0x7f0e40141a40 addr v2:172.21.15.83:3300/0 2023-10-06T13:52:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e40140dd0 con 0x7f0e400b3d60 2023-10-06T13:52:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e400aebe0 con 0x7f0e40141a40 2023-10-06T13:52:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e40141070 con 0x7f0e400b3990 2023-10-06T13:52:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.666+0000 7f0e474e5640 10 monclient(hunting): authenticate will time out at 1270.999045s 2023-10-06T13:52:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.667+0000 7f0e44a80640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0e400b3d60 0x7f0e40141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.667+0000 7f0e45a82640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e40141a40 0x7f0e400b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.667+0000 7f0e45281640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0e400b3990 0x7f0e400af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.667+0000 7f0e44a80640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0e400b3d60 0x7f0e40141500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.83:55220/0 (socket says 172.21.15.83:55220) 2023-10-06T13:52:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.667+0000 7f0e44a80640 1 -- 172.21.15.83:0/1229307134 learned_addr learned my addr 172.21.15.83:0/1229307134 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-10-06T13:52:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.667+0000 7f0e44a80640 10 monclient(hunting): get_auth_request con 0x7f0e400b3d60 auth_method 0 2023-10-06T13:52:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.667+0000 7f0e44a80640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.667+0000 7f0e44a80640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.667+0000 7f0e44a80640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.667+0000 7f0e45a82640 10 monclient(hunting): get_auth_request con 0x7f0e40141a40 auth_method 0 2023-10-06T13:52:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.667+0000 7f0e45a82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.667+0000 7f0e45a82640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.667+0000 7f0e45a82640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.668+0000 7f0e45281640 10 monclient(hunting): get_auth_request con 0x7f0e400b3990 auth_method 0 2023-10-06T13:52:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.668+0000 7f0e45281640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.668+0000 7f0e45281640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.668+0000 7f0e45281640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.668+0000 7f0e45a82640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.668+0000 7f0e45a82640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.668+0000 7f0e45a82640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.668+0000 7f0e44a80640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.668+0000 7f0e44a80640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.668+0000 7f0e44a80640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.668+0000 7f0e45281640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:22.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.668+0000 7f0e45281640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:22.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.668+0000 7f0e45281640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:22.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.670+0000 7f0e45a82640 10 monclient(hunting): handle_auth_done global_id 4117 payload 274 2023-10-06T13:52:22.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.670+0000 7f0e45a82640 1 -- 172.21.15.83:0/1229307134 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0e400b3990 msgr2=0x7f0e400af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:22.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.670+0000 7f0e45a82640 1 --2- 172.21.15.83:0/1229307134 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0e400b3990 0x7f0e400af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.670+0000 7f0e45a82640 1 -- 172.21.15.83:0/1229307134 >> v2:172.21.15.19:3300/0 conn(0x7f0e400b3d60 msgr2=0x7f0e40141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:22.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.670+0000 7f0e45a82640 1 --2- 172.21.15.83:0/1229307134 >> v2:172.21.15.19:3300/0 conn(0x7f0e400b3d60 0x7f0e40141500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.670+0000 7f0e45a82640 10 monclient: _finish_hunting 0 2023-10-06T13:52:22.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.670+0000 7f0e45a82640 1 monclient: found mon.noname-c 2023-10-06T13:52:22.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.670+0000 7f0e45a82640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:22.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.670+0000 7f0e45a82640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:52:22.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.670+0000 7f0e45a82640 1 -- 172.21.15.83:0/1229307134 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e400aee30 con 0x7f0e40141a40 2023-10-06T13:52:22.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.672+0000 7f0e45a82640 1 --2- 172.21.15.83:0/1229307134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e40141a40 0x7f0e400b00c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e400b1080 tx=0x7f0e3802bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5678942eb2b041e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:22.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e37fff640 1 -- 172.21.15.83:0/1229307134 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3802cb40 con 0x7f0e40141a40 2023-10-06T13:52:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e37fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e37fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:52:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e37fff640 10 monclient: dump: 2023-10-06T13:52:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:22.678 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e37fff640 10 monclient: _finish_auth 0 2023-10-06T13:52:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e37fff640 10 monclient: _check_auth_tickets 2023-10-06T13:52:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e37fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e37fff640 1 -- 172.21.15.83:0/1229307134 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3802cca0 con 0x7f0e40141a40 2023-10-06T13:52:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e37fff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e37fff640 1 -- 172.21.15.83:0/1229307134 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38034660 con 0x7f0e40141a40 2023-10-06T13:52:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e474e5640 5 monclient: authenticate success, global_id 4117 2023-10-06T13:52:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e474e5640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e474e5640 4 set_mon_vals no callback set 2023-10-06T13:52:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e37fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e37fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.675+0000 7f0e37fff640 10 monclient: dump: 2023-10-06T13:52:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:22.679 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.676+0000 7f0e474e5640 10 monclient: shutdown 2023-10-06T13:52:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.676+0000 7f0e474e5640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.676+0000 7f0e474e5640 1 -- 172.21.15.83:0/1229307134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e40141a40 msgr2=0x7f0e400b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.676+0000 7f0e474e5640 1 --2- 172.21.15.83:0/1229307134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e40141a40 0x7f0e400b00c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e400b1080 tx=0x7f0e3802bd90 comp rx=0 tx=0).stop 2023-10-06T13:52:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.676+0000 7f0e474e5640 1 -- 172.21.15.83:0/1229307134 shutdown_connections 2023-10-06T13:52:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.676+0000 7f0e474e5640 1 --2- 172.21.15.83:0/1229307134 >> v2:172.21.15.19:3300/0 conn(0x7f0e400b3d60 0x7f0e40141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.676+0000 7f0e474e5640 1 --2- 172.21.15.83:0/1229307134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e40141a40 0x7f0e400b00c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.676+0000 7f0e474e5640 1 --2- 172.21.15.83:0/1229307134 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0e400b3990 0x7f0e400af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.676+0000 7f0e474e5640 1 -- 172.21.15.83:0/1229307134 >> 172.21.15.83:0/1229307134 conn(0x7f0e400b6e00 msgr2=0x7f0e400b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.676+0000 7f0e474e5640 1 -- 172.21.15.83:0/1229307134 shutdown_connections 2023-10-06T13:52:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.676+0000 7f0e474e5640 1 -- 172.21.15.83:0/1229307134 wait complete. 2023-10-06T13:52:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.677+0000 7f0e474e5640 10 monclient: build_initial_monmap 2023-10-06T13:52:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.677+0000 7f0e474e5640 10 monclient: monmap: 2023-10-06T13:52:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 0.000000 2023-10-06T13:52:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:created 0.000000 2023-10-06T13:52:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:22.685 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.677+0000 7f0e474e5640 1 librados: starting msgr at 2023-10-06T13:52:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.677+0000 7f0e474e5640 1 librados: starting objecter 2023-10-06T13:52:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.677+0000 7f0e474e5640 1 Processor -- start 2023-10-06T13:52:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 1 -- start start 2023-10-06T13:52:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 1 librados: setting wanted keys 2023-10-06T13:52:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 1 librados: calling monclient init 2023-10-06T13:52:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 10 monclient: init 2023-10-06T13:52:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T13:52:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e400b3990 0x7f0e40144570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 10 monclient(hunting): picked mon.noname-b con 0x7f0e400b3990 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0e400b3d60 0x7f0e400aca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 10 monclient(hunting): picked mon.noname-c con 0x7f0e400b3d60 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e40141a40 0x7f0e401d6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 10 monclient(hunting): picked mon.noname-a con 0x7f0e40141a40 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e400ad150 con 0x7f0e40141a40 2023-10-06T13:52:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4013ac10 con 0x7f0e400b3990 2023-10-06T13:52:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4013a100 con 0x7f0e400b3d60 2023-10-06T13:52:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:22.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.678+0000 7f0e474e5640 10 monclient(hunting): authenticate will time out at 1271.011195s 2023-10-06T13:52:22.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.679+0000 7f0e45281640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e400b3990 0x7f0e40144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:22.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.679+0000 7f0e45281640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e400b3990 0x7f0e40144570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48340/0 (socket says 172.21.15.83:48340) 2023-10-06T13:52:22.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.679+0000 7f0e45281640 1 -- 172.21.15.83:0/3331847173 learned_addr learned my addr 172.21.15.83:0/3331847173 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-10-06T13:52:22.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.679+0000 7f0e45281640 10 monclient(hunting): get_auth_request con 0x7f0e400b3990 auth_method 0 2023-10-06T13:52:22.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.679+0000 7f0e45281640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:22.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.679+0000 7f0e45281640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:22.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.679+0000 7f0e45281640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:22.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.679+0000 7f0e45a82640 1 --2- 172.21.15.83:0/3331847173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e40141a40 0x7f0e401d6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:22.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.679+0000 7f0e44a80640 1 --2- 172.21.15.83:0/3331847173 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0e400b3d60 0x7f0e400aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:22.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e45a82640 10 monclient(hunting): get_auth_request con 0x7f0e40141a40 auth_method 0 2023-10-06T13:52:22.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e45a82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:22.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e45a82640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:22.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e45a82640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:22.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e45281640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:22.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e45281640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:22.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e45281640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:22.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e44a80640 10 monclient(hunting): get_auth_request con 0x7f0e400b3d60 auth_method 0 2023-10-06T13:52:22.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e44a80640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:22.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e44a80640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:22.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e44a80640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:22.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e45a82640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:22.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e45a82640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:22.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e45a82640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:22.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e44a80640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:22.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e44a80640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:22.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.680+0000 7f0e44a80640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.682+0000 7f0e45281640 10 monclient(hunting): handle_auth_done global_id 4120 payload 824 2023-10-06T13:52:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.682+0000 7f0e45281640 1 -- 172.21.15.83:0/3331847173 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0e400b3d60 msgr2=0x7f0e400aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.682+0000 7f0e45281640 1 --2- 172.21.15.83:0/3331847173 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0e400b3d60 0x7f0e400aca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.682+0000 7f0e45281640 1 -- 172.21.15.83:0/3331847173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e40141a40 msgr2=0x7f0e401d6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.682+0000 7f0e45281640 1 --2- 172.21.15.83:0/3331847173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e40141a40 0x7f0e401d6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.682+0000 7f0e45281640 10 monclient: _finish_hunting 0 2023-10-06T13:52:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.682+0000 7f0e45281640 1 monclient: found mon.noname-b 2023-10-06T13:52:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.682+0000 7f0e45281640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.682+0000 7f0e45281640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:52:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.682+0000 7f0e45281640 1 -- 172.21.15.83:0/3331847173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e38009a70 con 0x7f0e400b3990 2023-10-06T13:52:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.684+0000 7f0e45281640 1 --2- 172.21.15.83:0/3331847173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e400b3990 0x7f0e40144570 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00e890 tx=0x7f0e3c00ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.684+0000 7f0e35ffb640 10 client.?.objecter ms_handle_connect 0x7f0e400b3990 2023-10-06T13:52:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.685+0000 7f0e35ffb640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e35ffb640 1 -- 172.21.15.83:0/3331847173 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c00c940 con 0x7f0e400b3990 2023-10-06T13:52:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e35ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e35ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:52:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e35ffb640 10 monclient: dump: 2023-10-06T13:52:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:22.693 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e35ffb640 10 monclient: _finish_auth 0 2023-10-06T13:52:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e35ffb640 10 monclient: _check_auth_tickets 2023-10-06T13:52:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e35ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e35ffb640 1 -- 172.21.15.83:0/3331847173 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3c00caa0 con 0x7f0e400b3990 2023-10-06T13:52:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e35ffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e35ffb640 1 -- 172.21.15.83:0/3331847173 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c00ce30 con 0x7f0e400b3990 2023-10-06T13:52:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e474e5640 5 monclient: authenticate success, global_id 4120 2023-10-06T13:52:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e474e5640 10 monclient: _renew_subs 2023-10-06T13:52:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e474e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e474e5640 1 -- 172.21.15.83:0/3331847173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4013a380 con 0x7f0e400b3990 2023-10-06T13:52:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e35ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e377fe640 4 set_mon_vals no callback set 2023-10-06T13:52:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e35ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e35ffb640 10 monclient: dump: 2023-10-06T13:52:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:22.695 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e474e5640 10 client.4120.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e474e5640 10 monclient: _renew_subs 2023-10-06T13:52:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e474e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e474e5640 1 -- 172.21.15.83:0/3331847173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e401d6a90 con 0x7f0e400b3990 2023-10-06T13:52:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.688+0000 7f0e474e5640 1 librados: init done 2023-10-06T13:52:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.689+0000 7f0e474e5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.689+0000 7f0e474e5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.689+0000 7f0e474e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.689+0000 7f0e474e5640 1 -- 172.21.15.83:0/3331847173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e400678e0 con 0x7f0e400b3990 2023-10-06T13:52:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.692+0000 7f0e35ffb640 1 -- 172.21.15.83:0/3331847173 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0e3c005c90 con 0x7f0e400b3990 2023-10-06T13:52:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.692+0000 7f0e35ffb640 1 -- 172.21.15.83:0/3331847173 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f0e3c006880 con 0x7f0e400b3990 2023-10-06T13:52:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.692+0000 7f0e35ffb640 10 client.4120.objecter ms_dispatch 0x7f0e40000b90 osd_map(4..4 src has 1..4) v4 2023-10-06T13:52:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.693+0000 7f0e35ffb640 3 client.4120.objecter handle_osd_map got epochs [4,4] > 0 2023-10-06T13:52:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.693+0000 7f0e35ffb640 3 client.4120.objecter handle_osd_map decoding full epoch 4 2023-10-06T13:52:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.693+0000 7f0e35ffb640 20 client.4120.objecter dump_active .. 0 homeless 2023-10-06T13:52:22.727 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:22.726+0000 7f6f00a18040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T13:52:22.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.789+0000 7f0e35ffb640 1 -- 172.21.15.83:0/3331847173 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f0e3c021060 con 0x7f0e400b3990 2023-10-06T13:52:22.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.789+0000 7f0e35ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:22.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.789+0000 7f0e35ffb640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:22.866 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:22.865+0000 7f6f00a18040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T13:52:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.880+0000 7f0e474e5640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "06bcfab5-d2f1-4447-ba44-143398707ef4", "id": 2}] 2023-10-06T13:52:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.880+0000 7f0e474e5640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "06bcfab5-d2f1-4447-ba44-143398707ef4", "id": 2}] 2023-10-06T13:52:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.880+0000 7f0e474e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:22.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.880+0000 7f0e474e5640 1 -- 172.21.15.83:0/3331847173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd new", "uuid": "06bcfab5-d2f1-4447-ba44-143398707ef4", "id": 2} v 0) v1 -- 0x7f0e400ae200 con 0x7f0e400b3990 2023-10-06T13:52:22.894 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:22.892+0000 7f4569a0e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T13:52:22.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.966+0000 7f0e35ffb640 1 -- 172.21.15.83:0/3331847173 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "06bcfab5-d2f1-4447-ba44-143398707ef4", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0e3c017070 con 0x7f0e400b3990 2023-10-06T13:52:22.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.966+0000 7f0e35ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "06bcfab5-d2f1-4447-ba44-143398707ef4", "id": 2}] 2023-10-06T13:52:22.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.966+0000 7f0e35ffb640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:52:22.969 INFO:teuthology.orchestra.run.smithi083.stdout:2 2023-10-06T13:52:22.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.967+0000 7f0e474e5640 10 librados: watch_flush enter 2023-10-06T13:52:22.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.967+0000 7f0e474e5640 10 librados: watch_flush exit 2023-10-06T13:52:22.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.967+0000 7f0e474e5640 20 client.4120.objecter shutdown clearing up homeless session... 2023-10-06T13:52:22.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.967+0000 7f0e474e5640 10 client.4120.objecter successfully canceled tick 2023-10-06T13:52:22.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.967+0000 7f0e474e5640 10 monclient: shutdown 2023-10-06T13:52:22.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.967+0000 7f0e474e5640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:22.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.967+0000 7f0e474e5640 1 -- 172.21.15.83:0/3331847173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e400b3990 msgr2=0x7f0e40144570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:22.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.967+0000 7f0e474e5640 1 --2- 172.21.15.83:0/3331847173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e400b3990 0x7f0e40144570 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00e890 tx=0x7f0e3c00ed50 comp rx=0 tx=0).stop 2023-10-06T13:52:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.967+0000 7f0e474e5640 1 -- 172.21.15.83:0/3331847173 shutdown_connections 2023-10-06T13:52:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.967+0000 7f0e474e5640 1 --2- 172.21.15.83:0/3331847173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e40141a40 0x7f0e401d6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.967+0000 7f0e474e5640 1 --2- 172.21.15.83:0/3331847173 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0e400b3d60 0x7f0e400aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.967+0000 7f0e474e5640 1 --2- 172.21.15.83:0/3331847173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e400b3990 0x7f0e40144570 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.967+0000 7f0e474e5640 1 -- 172.21.15.83:0/3331847173 >> 172.21.15.83:0/3331847173 conn(0x7f0e400b6e00 msgr2=0x7f0e4013e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.967+0000 7f0e474e5640 1 -- 172.21.15.83:0/3331847173 shutdown_connections 2023-10-06T13:52:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.968+0000 7f0e474e5640 1 -- 172.21.15.83:0/3331847173 wait complete. 2023-10-06T13:52:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:22.968+0000 7f0e474e5640 1 librados: shutdown 2023-10-06T13:52:22.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new 7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5 3 2023-10-06T13:52:22.994 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:22.993+0000 7f6f00a18040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T13:52:23.037 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:23.035+0000 7f4569a0e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T13:52:23.042 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:23.041+0000 7f6f00a18040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T13:52:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.076+0000 7fddac674640 10 monclient: get_monmap_and_config 2023-10-06T13:52:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.076+0000 7fddac674640 10 monclient: build_initial_monmap 2023-10-06T13:52:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.076+0000 7fddac674640 10 monclient: monmap: 2023-10-06T13:52:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:52:23.076798+0000 2023-10-06T13:52:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:52:23.076798+0000 2023-10-06T13:52:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:52:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:52:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:52:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:52:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:52:23.078 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.076+0000 7fddac674640 1 Processor -- start 2023-10-06T13:52:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.076+0000 7fddac674640 1 -- start start 2023-10-06T13:52:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.076+0000 7fddac674640 10 monclient: init 2023-10-06T13:52:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.077+0000 7fddac674640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.077+0000 7fddac674640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2023-10-06T13:52:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.077+0000 7fddac674640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fdda406b6d0 addr v1:172.21.15.83:6789/0 2023-10-06T13:52:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.077+0000 7fddac674640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdda4144910 0x7fdda413df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.077+0000 7fddac674640 10 monclient(hunting): picked mon.noname-a con 0x7fdda4144910 addr v2:172.21.15.19:3300/0 2023-10-06T13:52:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.077+0000 7fddac674640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdda413e460 0x7fdda414b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.077+0000 7fddac674640 10 monclient(hunting): picked mon.noname-b con 0x7fdda413e460 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.077+0000 7fddac674640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda406baa0 con 0x7fdda406b6d0 2023-10-06T13:52:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.077+0000 7fddac674640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.077+0000 7fddac674640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda40b2530 con 0x7fdda4144910 2023-10-06T13:52:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.077+0000 7fddac674640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.077+0000 7fddac674640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda40b2810 con 0x7fdda413e460 2023-10-06T13:52:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.077+0000 7fddac674640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.077+0000 7fddac674640 10 monclient(hunting): authenticate will time out at 1271.409785s 2023-10-06T13:52:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.078+0000 7fddaa410640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdda406b6d0 0x7fdda4144200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56676/0 (socket says 172.21.15.83:56676) 2023-10-06T13:52:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.078+0000 7fddaa410640 1 -- 172.21.15.83:0/3681791766 learned_addr learned my addr 172.21.15.83:0/3681791766 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-10-06T13:52:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.078+0000 7fdda9c0f640 1 --2- 172.21.15.83:0/3681791766 >> v2:172.21.15.19:3300/0 conn(0x7fdda4144910 0x7fdda413df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.078+0000 7fddaac11640 1 --2- 172.21.15.83:0/3681791766 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdda413e460 0x7fdda414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.078+0000 7fdda9c0f640 10 monclient(hunting): get_auth_request con 0x7fdda4144910 auth_method 0 2023-10-06T13:52:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.078+0000 7fdda9c0f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.078+0000 7fdda9c0f640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.078+0000 7fdda9c0f640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.079+0000 7fddaac11640 10 monclient(hunting): get_auth_request con 0x7fdda413e460 auth_method 0 2023-10-06T13:52:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.079+0000 7fddaac11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.079+0000 7fddaac11640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.079+0000 7fddaac11640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.079+0000 7fdda9c0f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.079+0000 7fdda9c0f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.079+0000 7fdda9c0f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.079+0000 7fddaac11640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.079+0000 7fddaac11640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.079+0000 7fddaac11640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.081+0000 7fdda9c0f640 10 monclient(hunting): handle_auth_done global_id 4131 payload 274 2023-10-06T13:52:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.081+0000 7fdda9c0f640 1 -- 172.21.15.83:0/3681791766 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdda413e460 msgr2=0x7fdda414b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.081+0000 7fdda9c0f640 1 --2- 172.21.15.83:0/3681791766 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdda413e460 0x7fdda414b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.081+0000 7fdda9c0f640 1 -- 172.21.15.83:0/3681791766 >> v1:172.21.15.83:6789/0 conn(0x7fdda406b6d0 legacy=0x7fdda4144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.081+0000 7fdda9c0f640 10 monclient: _finish_hunting 0 2023-10-06T13:52:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.081+0000 7fdda9c0f640 1 monclient: found mon.noname-a 2023-10-06T13:52:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.081+0000 7fdda9c0f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.081+0000 7fdda9c0f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:52:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.081+0000 7fdda9c0f640 1 -- 172.21.15.83:0/3681791766 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda40aeff0 con 0x7fdda4144910 2023-10-06T13:52:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.083+0000 7fdda9c0f640 1 --2- 172.21.15.83:0/3681791766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdda4144910 0x7fdda413df20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdd940097a0 tx=0x7fdd9402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e5232ba93e162 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fdda940e640 1 -- 172.21.15.83:0/3681791766 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd94031830 con 0x7fdda4144910 2023-10-06T13:52:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fdda940e640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fdda940e640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:52:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fdda940e640 10 monclient: dump: 2023-10-06T13:52:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:23.090 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fdda940e640 10 monclient: _finish_auth 0 2023-10-06T13:52:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fdda940e640 10 monclient: _check_auth_tickets 2023-10-06T13:52:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fdda940e640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fddac674640 5 monclient: authenticate success, global_id 4131 2023-10-06T13:52:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fddac674640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:52:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fdda940e640 1 -- 172.21.15.83:0/3681791766 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd94031990 con 0x7fdda4144910 2023-10-06T13:52:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fdda940e640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fddac674640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fddac674640 4 set_mon_vals no callback set 2023-10-06T13:52:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fdda940e640 1 -- 172.21.15.83:0/3681791766 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd94031d20 con 0x7fdda4144910 2023-10-06T13:52:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fdda940e640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fdda940e640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:52:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fdda940e640 10 monclient: dump: 2023-10-06T13:52:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:23.092 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fddac674640 10 monclient: shutdown 2023-10-06T13:52:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fddac674640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fddac674640 1 -- 172.21.15.83:0/3681791766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdda4144910 msgr2=0x7fdda413df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fddac674640 1 --2- 172.21.15.83:0/3681791766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdda4144910 0x7fdda413df20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdd940097a0 tx=0x7fdd9402b8d0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fddac674640 1 -- 172.21.15.83:0/3681791766 shutdown_connections 2023-10-06T13:52:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fddac674640 1 --2- 172.21.15.83:0/3681791766 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdda413e460 0x7fdda414b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fddac674640 1 --2- 172.21.15.83:0/3681791766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdda4144910 0x7fdda413df20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fddac674640 1 -- 172.21.15.83:0/3681791766 >> 172.21.15.83:0/3681791766 conn(0x7fdda40665b0 msgr2=0x7fdda406ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.087+0000 7fddac674640 1 -- 172.21.15.83:0/3681791766 shutdown_connections 2023-10-06T13:52:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.088+0000 7fddac674640 1 -- 172.21.15.83:0/3681791766 wait complete. 2023-10-06T13:52:23.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.088+0000 7fddac674640 10 monclient: build_initial_monmap 2023-10-06T13:52:23.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.088+0000 7fddac674640 10 monclient: monmap: 2023-10-06T13:52:23.096 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 0.000000 2023-10-06T13:52:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:created 0.000000 2023-10-06T13:52:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:23.097 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.088+0000 7fddac674640 1 librados: starting msgr at 2023-10-06T13:52:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.088+0000 7fddac674640 1 librados: starting objecter 2023-10-06T13:52:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.088+0000 7fddac674640 1 Processor -- start 2023-10-06T13:52:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 1 -- start start 2023-10-06T13:52:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 1 librados: setting wanted keys 2023-10-06T13:52:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 1 librados: calling monclient init 2023-10-06T13:52:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 10 monclient: init 2023-10-06T13:52:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T13:52:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda406b6d0 0x7fdda41caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 10 monclient(hunting): picked mon.noname-b con 0x7fdda406b6d0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdda413e460 0x7fdda41caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 10 monclient(hunting): picked mon.noname-a con 0x7fdda413e460 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdda4144910 0x7fdda41ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 10 monclient(hunting): picked mon.noname-c con 0x7fdda4144910 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda41cc870 con 0x7fdda413e460 2023-10-06T13:52:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda41d3240 con 0x7fdda406b6d0 2023-10-06T13:52:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda41c9ca0 con 0x7fdda4144910 2023-10-06T13:52:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.089+0000 7fddac674640 10 monclient(hunting): authenticate will time out at 1271.421846s 2023-10-06T13:52:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.090+0000 7fdda9c0f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdda413e460 0x7fdda41caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.090+0000 7fddaac11640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdda4144910 0x7fdda41ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.090+0000 7fdda9c0f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdda413e460 0x7fdda41caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.83:55236/0 (socket says 172.21.15.83:55236) 2023-10-06T13:52:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.090+0000 7fdda9c0f640 1 -- 172.21.15.83:0/804052220 learned_addr learned my addr 172.21.15.83:0/804052220 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-10-06T13:52:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.090+0000 7fdda9c0f640 10 monclient(hunting): get_auth_request con 0x7fdda413e460 auth_method 0 2023-10-06T13:52:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.090+0000 7fdda9c0f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.090+0000 7fdda9c0f640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.090+0000 7fdda9c0f640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.090+0000 7fddaac11640 10 monclient(hunting): get_auth_request con 0x7fdda4144910 auth_method 0 2023-10-06T13:52:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.090+0000 7fddaac11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.090+0000 7fddaac11640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.090+0000 7fddaac11640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.091+0000 7fdda9c0f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.091+0000 7fdda9c0f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.091+0000 7fdda9c0f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.091+0000 7fddaac11640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.091+0000 7fddaac11640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.091+0000 7fddaac11640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.091+0000 7fddaa410640 1 --2- 172.21.15.83:0/804052220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda406b6d0 0x7fdda41caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.092+0000 7fddaa410640 10 monclient(hunting): get_auth_request con 0x7fdda406b6d0 auth_method 0 2023-10-06T13:52:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.092+0000 7fddaa410640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.092+0000 7fddaa410640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.092+0000 7fddaa410640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.092+0000 7fddaa410640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.092+0000 7fddaa410640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.092+0000 7fddaa410640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.094+0000 7fdda9c0f640 10 monclient(hunting): handle_auth_done global_id 4134 payload 824 2023-10-06T13:52:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.094+0000 7fdda9c0f640 1 -- 172.21.15.83:0/804052220 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdda4144910 msgr2=0x7fdda41ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.094+0000 7fdda9c0f640 1 --2- 172.21.15.83:0/804052220 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdda4144910 0x7fdda41ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.094+0000 7fdda9c0f640 1 -- 172.21.15.83:0/804052220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda406b6d0 msgr2=0x7fdda41caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.094+0000 7fdda9c0f640 1 --2- 172.21.15.83:0/804052220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda406b6d0 0x7fdda41caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.094+0000 7fdda9c0f640 10 monclient: _finish_hunting 0 2023-10-06T13:52:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.094+0000 7fdda9c0f640 1 monclient: found mon.noname-a 2023-10-06T13:52:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.094+0000 7fdda9c0f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.094+0000 7fdda9c0f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:52:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.094+0000 7fdda9c0f640 1 -- 172.21.15.83:0/804052220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd94009400 con 0x7fdda413e460 2023-10-06T13:52:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.096+0000 7fdda9c0f640 1 --2- 172.21.15.83:0/804052220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdda413e460 0x7fdda41caf60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdd94034040 tx=0x7fdd940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.096+0000 7fdd9affd640 10 client.?.objecter ms_handle_connect 0x7fdda413e460 2023-10-06T13:52:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.096+0000 7fdd9affd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fdd9affd640 1 -- 172.21.15.83:0/804052220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9403a8f0 con 0x7fdda413e460 2023-10-06T13:52:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fdd9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fdd9affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:52:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fdd9affd640 10 monclient: dump: 2023-10-06T13:52:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:23.105 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fdd9affd640 10 monclient: _finish_auth 0 2023-10-06T13:52:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fdd9affd640 10 monclient: _check_auth_tickets 2023-10-06T13:52:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fdd9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fddac674640 5 monclient: authenticate success, global_id 4134 2023-10-06T13:52:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fddac674640 10 monclient: _renew_subs 2023-10-06T13:52:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fddac674640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:52:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fddac674640 1 -- 172.21.15.83:0/804052220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda41c9ec0 con 0x7fdda413e460 2023-10-06T13:52:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fddac674640 10 client.4134.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fddac674640 10 monclient: _renew_subs 2023-10-06T13:52:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fddac674640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:52:23.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fddac674640 1 -- 172.21.15.83:0/804052220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda41cd400 con 0x7fdda413e460 2023-10-06T13:52:23.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.099+0000 7fddac674640 1 librados: init done 2023-10-06T13:52:23.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.100+0000 7fdd9affd640 1 -- 172.21.15.83:0/804052220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9403aa50 con 0x7fdda413e460 2023-10-06T13:52:23.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.100+0000 7fdd9affd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:23.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.100+0000 7fdd9affd640 1 -- 172.21.15.83:0/804052220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9403adc0 con 0x7fdda413e460 2023-10-06T13:52:23.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.100+0000 7fdd9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:23.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.100+0000 7fdd9affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:52:23.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.100+0000 7fdda8c0d640 4 set_mon_vals no callback set 2023-10-06T13:52:23.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.100+0000 7fdd9affd640 10 monclient: dump: 2023-10-06T13:52:23.106 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:23.106 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.106 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.106 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.107 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:23.107 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.107 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:23.107 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:23.107 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:23.107 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.100+0000 7fddac674640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:23.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.101+0000 7fddac674640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:23.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.101+0000 7fddac674640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:52:23.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.101+0000 7fddac674640 1 -- 172.21.15.83:0/804052220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8c005040 con 0x7fdda413e460 2023-10-06T13:52:23.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.103+0000 7fdd9affd640 1 -- 172.21.15.83:0/804052220 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdd9404d740 con 0x7fdda413e460 2023-10-06T13:52:23.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.103+0000 7fdd9affd640 1 -- 172.21.15.83:0/804052220 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fdd94044a70 con 0x7fdda413e460 2023-10-06T13:52:23.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.103+0000 7fdd9affd640 10 client.4134.objecter ms_dispatch 0x7fdda4000b90 osd_map(5..5 src has 1..5) v4 2023-10-06T13:52:23.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.103+0000 7fdd9affd640 3 client.4134.objecter handle_osd_map got epochs [5,5] > 0 2023-10-06T13:52:23.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.103+0000 7fdd9affd640 3 client.4134.objecter handle_osd_map decoding full epoch 5 2023-10-06T13:52:23.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.103+0000 7fdd9affd640 20 client.4134.objecter dump_active .. 0 homeless 2023-10-06T13:52:23.148 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:23.147+0000 7f6f00a18040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T13:52:23.161 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:23.159+0000 7f4569a0e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T13:52:23.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.206+0000 7fdd9affd640 1 -- 172.21.15.83:0/804052220 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fdd94035020 con 0x7fdda413e460 2023-10-06T13:52:23.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.206+0000 7fdd9affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:23.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.206+0000 7fdd9affd640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:23.208 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:23.206+0000 7f4569a0e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T13:52:23.243 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:23.242+0000 7f6f00a18040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T13:52:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.278+0000 7fddac674640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5", "id": 3}] 2023-10-06T13:52:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.278+0000 7fddac674640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5", "id": 3}] 2023-10-06T13:52:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.278+0000 7fddac674640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:52:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.278+0000 7fddac674640 1 -- 172.21.15.83:0/804052220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5", "id": 3} v 0) v1 -- 0x7fdd8c0055c0 con 0x7fdda413e460 2023-10-06T13:52:23.313 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:23.311+0000 7f4569a0e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T13:52:23.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.327+0000 7fdd9affd640 1 -- 172.21.15.83:0/804052220 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdd94044d60 con 0x7fdda413e460 2023-10-06T13:52:23.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.327+0000 7fdd9affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5", "id": 3}] 2023-10-06T13:52:23.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.327+0000 7fdd9affd640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:52:23.329 INFO:teuthology.orchestra.run.smithi083.stdout:3 2023-10-06T13:52:23.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 10 librados: watch_flush enter 2023-10-06T13:52:23.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 10 librados: watch_flush exit 2023-10-06T13:52:23.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 20 client.4134.objecter shutdown clearing up homeless session... 2023-10-06T13:52:23.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 10 client.4134.objecter successfully canceled tick 2023-10-06T13:52:23.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 10 monclient: shutdown 2023-10-06T13:52:23.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:23.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 1 -- 172.21.15.83:0/804052220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdda413e460 msgr2=0x7fdda41caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 1 --2- 172.21.15.83:0/804052220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdda413e460 0x7fdda41caf60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdd94034040 tx=0x7fdd940046c0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 1 -- 172.21.15.83:0/804052220 shutdown_connections 2023-10-06T13:52:23.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 1 --2- 172.21.15.83:0/804052220 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdda4144910 0x7fdda41ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 1 --2- 172.21.15.83:0/804052220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdda413e460 0x7fdda41caf60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 1 --2- 172.21.15.83:0/804052220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda406b6d0 0x7fdda41caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 1 -- 172.21.15.83:0/804052220 >> 172.21.15.83:0/804052220 conn(0x7fdda40665b0 msgr2=0x7fdda40b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:23.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 1 -- 172.21.15.83:0/804052220 shutdown_connections 2023-10-06T13:52:23.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 1 -- 172.21.15.83:0/804052220 wait complete. 2023-10-06T13:52:23.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.328+0000 7fddac674640 1 librados: shutdown 2023-10-06T13:52:23.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new ad31d4df-e78c-4e32-bfd4-3fe6e70dd428 4 2023-10-06T13:52:23.386 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:23.385+0000 7f6f00a18040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T13:52:23.413 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:23.411+0000 7f4569a0e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T13:52:23.431 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:23.430+0000 7f6f00a18040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T13:52:23.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.480+0000 7faed7084640 10 monclient: get_monmap_and_config 2023-10-06T13:52:23.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.480+0000 7faed7084640 10 monclient: build_initial_monmap 2023-10-06T13:52:23.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.480+0000 7faed7084640 10 monclient: monmap: 2023-10-06T13:52:23.482 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:23.482 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.482 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:52:23.480979+0000 2023-10-06T13:52:23.482 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:52:23.480979+0000 2023-10-06T13:52:23.482 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:23.482 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.482 INFO:teuthology.orchestra.run.smithi083.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:52:23.482 INFO:teuthology.orchestra.run.smithi083.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:52:23.482 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:52:23.483 INFO:teuthology.orchestra.run.smithi083.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:52:23.483 INFO:teuthology.orchestra.run.smithi083.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:52:23.483 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 1 Processor -- start 2023-10-06T13:52:23.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 1 -- start start 2023-10-06T13:52:23.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 10 monclient: init 2023-10-06T13:52:23.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:23.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 10 monclient: _add_conns ranks=[0,1,4,2,3] 2023-10-06T13:52:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faed00b3990 addr v1:172.21.15.19:6789/0 2023-10-06T13:52:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faed00b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T13:52:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faed00b07d0 0x7faed00af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 10 monclient(hunting): picked mon.noname-c con 0x7faed00b07d0 addr v2:172.21.15.83:3300/0 2023-10-06T13:52:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed00aebe0 con 0x7faed00b3990 2023-10-06T13:52:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed0141070 con 0x7faed00b3d60 2023-10-06T13:52:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed0140dd0 con 0x7faed00b07d0 2023-10-06T13:52:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.481+0000 7faed7084640 10 monclient(hunting): authenticate will time out at 1271.813940s 2023-10-06T13:52:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.482+0000 7faed5621640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faed00b07d0 0x7faed00af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:23.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.482+0000 7faed4e20640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7faed00b3990 0x7faed0141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.83:43626/0 (socket says 172.21.15.83:43626) 2023-10-06T13:52:23.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.482+0000 7faed4e20640 1 -- 172.21.15.83:0/1768195482 learned_addr learned my addr 172.21.15.83:0/1768195482 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-10-06T13:52:23.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.482+0000 7faed5621640 10 monclient(hunting): get_auth_request con 0x7faed00b07d0 auth_method 0 2023-10-06T13:52:23.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.482+0000 7faed5621640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:23.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.482+0000 7faed5621640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.482+0000 7faed5621640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.483+0000 7faed5621640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:23.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.483+0000 7faed5621640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:23.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.483+0000 7faed5621640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:23.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.484+0000 7faecf7fe640 1 -- 172.21.15.83:0/1768195482 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989772805 0 0) 0x7faed00aebe0 con 0x7faed00b3990 2023-10-06T13:52:23.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.484+0000 7faecf7fe640 10 monclient(hunting): _negotiate 2023-10-06T13:52:23.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.484+0000 7faecf7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.484+0000 7faecf7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.484+0000 7faecf7fe640 10 monclient(hunting): my global_id is 4137 2023-10-06T13:52:23.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.484+0000 7faecf7fe640 1 -- 172.21.15.83:0/1768195482 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faec0003520 con 0x7faed00b3990 2023-10-06T13:52:23.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 -- 172.21.15.83:0/1768195482 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 180544589 0 0) 0x7faec0003520 con 0x7faed00b3990 2023-10-06T13:52:23.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 -- 172.21.15.83:0/1768195482 >> v2:172.21.15.83:3300/0 conn(0x7faed00b07d0 msgr2=0x7faed00af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:23.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 --2- 172.21.15.83:0/1768195482 >> v2:172.21.15.83:3300/0 conn(0x7faed00b07d0 0x7faed00af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 -- 172.21.15.83:0/1768195482 >> v1:172.21.15.83:6789/0 conn(0x7faed00b3d60 legacy=0x7faed00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient: _finish_hunting 0 2023-10-06T13:52:23.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 monclient: found mon.noname-a-legacy 2023-10-06T13:52:23.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:23.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.19:6789/0 2023-10-06T13:52:23.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 -- 172.21.15.83:0/1768195482 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed0141c10 con 0x7faed00b3990 2023-10-06T13:52:23.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient: _finish_auth 0 2023-10-06T13:52:23.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient: _check_auth_tickets 2023-10-06T13:52:23.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:23.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 -- 172.21.15.83:0/1768195482 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251531364 0 0) 0x7faec4001670 con 0x7faed00b3990 2023-10-06T13:52:23.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:23.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2023-10-06T13:52:23.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient: dump: 2023-10-06T13:52:23.492 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:23.492 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.492 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.492 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.492 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:23.492 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.492 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:23.493 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] but i'm connected to v1:172.21.15.19:6789/0, reconnecting 2023-10-06T13:52:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 -- 172.21.15.83:0/1768195482 >> v1:172.21.15.19:6789/0 conn(0x7faed00b3990 legacy=0x7faed0141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T13:52:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 --2- 172.21.15.83:0/1768195482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faec00039c0 0x7faec0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient(hunting): picked mon.b con 0x7faec00039c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 --2- 172.21.15.83:0/1768195482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faec00047c0 0x7faec0004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient(hunting): picked mon.a con 0x7faec00047c0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 --2- 172.21.15.83:0/1768195482 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faec00051c0 0x7faec000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient(hunting): picked mon.c con 0x7faec00051c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:23.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 -- 172.21.15.83:0/1768195482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed0140dd0 con 0x7faec00047c0 2023-10-06T13:52:23.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 -- 172.21.15.83:0/1768195482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec000c0d0 con 0x7faec00039c0 2023-10-06T13:52:23.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 1 -- 172.21.15.83:0/1768195482 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec000c290 con 0x7faec00051c0 2023-10-06T13:52:23.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:23.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.488+0000 7faecf7fe640 10 monclient(hunting): _finish_auth 0 2023-10-06T13:52:23.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.489+0000 7faed5621640 1 --2- 172.21.15.83:0/1768195482 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faec00051c0 0x7faec000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:23.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.489+0000 7faed5621640 10 monclient(hunting): get_auth_request con 0x7faec00051c0 auth_method 0 2023-10-06T13:52:23.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.489+0000 7faed5621640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:23.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.489+0000 7faed5621640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.489+0000 7faed5621640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:23.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.490+0000 7faed5621640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:23.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.490+0000 7faed5621640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:23.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.490+0000 7faed5621640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T13:52:23.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faecffff640 1 --2- 172.21.15.83:0/1768195482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faec00047c0 0x7faec0004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:23.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed5621640 10 monclient(hunting): handle_auth_done global_id 4137 payload 293 2023-10-06T13:52:23.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed5621640 1 -- 172.21.15.83:0/1768195482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faec00039c0 msgr2=0x7faec0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:23.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed5621640 1 --2- 172.21.15.83:0/1768195482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faec00039c0 0x7faec0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed5621640 1 -- 172.21.15.83:0/1768195482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faec00047c0 msgr2=0x7faec0004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:23.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed5621640 1 --2- 172.21.15.83:0/1768195482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faec00047c0 0x7faec0004c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed5621640 10 monclient: _finish_hunting 0 2023-10-06T13:52:23.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed5621640 1 monclient: found mon.c 2023-10-06T13:52:23.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed5621640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:23.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed5621640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:52:23.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed5621640 1 -- 172.21.15.83:0/1768195482 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec0003520 con 0x7faec00051c0 2023-10-06T13:52:23.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed5621640 10 monclient: _finish_auth 0 2023-10-06T13:52:23.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed5621640 10 monclient: _check_auth_tickets 2023-10-06T13:52:23.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed5621640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:23.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed7084640 5 monclient: authenticate success, global_id 4137 2023-10-06T13:52:23.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.492+0000 7faed7084640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:52:23.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.493+0000 7faed5621640 1 -- 172.21.15.83:0/1768195482 reap_dead start 2023-10-06T13:52:23.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.495+0000 7faed5621640 1 --2- 172.21.15.83:0/1768195482 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faec00051c0 0x7faec000b980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faec8009fc0 tx=0x7faec00057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee575d891a21eb37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:23.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.531+0000 7faecf7fe640 1 -- 172.21.15.83:0/1768195482 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec802cc60 con 0x7faec00051c0 2023-10-06T13:52:23.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.531+0000 7faecf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:23.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.531+0000 7faecf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:52:23.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.531+0000 7faecf7fe640 10 monclient: dump: 2023-10-06T13:52:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:23.534 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faecf7fe640 1 -- 172.21.15.83:0/1768195482 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec800f040 con 0x7faec00051c0 2023-10-06T13:52:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faecf7fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:23.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faecf7fe640 1 -- 172.21.15.83:0/1768195482 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8003990 con 0x7faec00051c0 2023-10-06T13:52:23.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faed7084640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:23.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faed7084640 4 set_mon_vals no callback set 2023-10-06T13:52:23.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faecf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:23.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faecf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:52:23.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faecf7fe640 10 monclient: dump: 2023-10-06T13:52:23.535 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:23.535 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.535 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.535 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.535 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:23.535 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:23.536 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faed7084640 10 monclient: shutdown 2023-10-06T13:52:23.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faed7084640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:23.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faed7084640 1 -- 172.21.15.83:0/1768195482 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faec00051c0 msgr2=0x7faec000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faed7084640 1 --2- 172.21.15.83:0/1768195482 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faec00051c0 0x7faec000b980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faec8009fc0 tx=0x7faec00057f0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faed7084640 1 -- 172.21.15.83:0/1768195482 shutdown_connections 2023-10-06T13:52:23.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faed7084640 1 --2- 172.21.15.83:0/1768195482 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faec00051c0 0x7faec000b980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faed7084640 1 -- 172.21.15.83:0/1768195482 >> 172.21.15.83:0/1768195482 conn(0x7faed00b6e00 msgr2=0x7faed00b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:23.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faed7084640 1 -- 172.21.15.83:0/1768195482 shutdown_connections 2023-10-06T13:52:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.532+0000 7faed7084640 1 -- 172.21.15.83:0/1768195482 wait complete. 2023-10-06T13:52:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.533+0000 7faed7084640 10 monclient: build_initial_monmap 2023-10-06T13:52:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.533+0000 7faed7084640 10 monclient: monmap: 2023-10-06T13:52:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 0.000000 2023-10-06T13:52:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:created 0.000000 2023-10-06T13:52:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:23.541 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.533+0000 7faed7084640 1 librados: starting msgr at 2023-10-06T13:52:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.533+0000 7faed7084640 1 librados: starting objecter 2023-10-06T13:52:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.533+0000 7faed7084640 1 Processor -- start 2023-10-06T13:52:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.533+0000 7faed7084640 1 -- start start 2023-10-06T13:52:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.533+0000 7faed7084640 1 librados: setting wanted keys 2023-10-06T13:52:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.533+0000 7faed7084640 1 librados: calling monclient init 2023-10-06T13:52:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.533+0000 7faed7084640 10 monclient: init 2023-10-06T13:52:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 10 monclient: _add_conns ranks=[2,1,0] 2023-10-06T13:52:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faec00051c0 0x7faed0144440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 10 monclient(hunting): picked mon.noname-c con 0x7faec00051c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed00aa470 0x7faed00a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 10 monclient(hunting): picked mon.noname-b con 0x7faed00aa470 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed01d5d10 0x7faed01d61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 10 monclient(hunting): picked mon.noname-a con 0x7faed01d5d10 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00aa8a0 con 0x7faed01d5d10 2023-10-06T13:52:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00aa290 con 0x7faed00aa470 2023-10-06T13:52:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed01d66f0 con 0x7faec00051c0 2023-10-06T13:52:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.534+0000 7faed7084640 10 monclient(hunting): authenticate will time out at 1271.866930s 2023-10-06T13:52:23.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.535+0000 7faecffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed00aa470 0x7faed00a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:23.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.535+0000 7faed5621640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed01d5d10 0x7faed01d61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:23.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.535+0000 7faecffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed00aa470 0x7faed00a9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48368/0 (socket says 172.21.15.83:48368) 2023-10-06T13:52:23.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.535+0000 7faecffff640 1 -- 172.21.15.83:0/3320723215 learned_addr learned my addr 172.21.15.83:0/3320723215 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-10-06T13:52:23.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.535+0000 7faecffff640 10 monclient(hunting): get_auth_request con 0x7faed00aa470 auth_method 0 2023-10-06T13:52:23.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.535+0000 7faecffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:23.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.535+0000 7faecffff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.535+0000 7faecffff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.535+0000 7faed5621640 10 monclient(hunting): get_auth_request con 0x7faed01d5d10 auth_method 0 2023-10-06T13:52:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.535+0000 7faed5621640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.535+0000 7faed5621640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.535+0000 7faed5621640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.536+0000 7faecffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.536+0000 7faecffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.536+0000 7faecffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.536+0000 7faed5621640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.536+0000 7faed5621640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.536+0000 7faed5621640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:23.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.538+0000 7faecffff640 10 monclient(hunting): handle_auth_done global_id 4135 payload 824 2023-10-06T13:52:23.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.538+0000 7faecffff640 1 -- 172.21.15.83:0/3320723215 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faec00051c0 msgr2=0x7faed0144440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.538+0000 7faecffff640 1 --2- 172.21.15.83:0/3320723215 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faec00051c0 0x7faed0144440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.538+0000 7faecffff640 1 -- 172.21.15.83:0/3320723215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed01d5d10 msgr2=0x7faed01d61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.538+0000 7faecffff640 1 --2- 172.21.15.83:0/3320723215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed01d5d10 0x7faed01d61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.538+0000 7faecffff640 10 monclient: _finish_hunting 0 2023-10-06T13:52:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.538+0000 7faecffff640 1 monclient: found mon.noname-b 2023-10-06T13:52:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.538+0000 7faecffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.538+0000 7faecffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:52:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.538+0000 7faecffff640 1 -- 172.21.15.83:0/3320723215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00aaa90 con 0x7faed00aa470 2023-10-06T13:52:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.540+0000 7faecffff640 1 --2- 172.21.15.83:0/3320723215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed00aa470 0x7faed00a9d50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faebc00d4f0 tx=0x7faebc00d9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.540+0000 7faecd7fa640 10 client.?.objecter ms_handle_connect 0x7faed00aa470 2023-10-06T13:52:23.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.540+0000 7faecd7fa640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:23.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faecd7fa640 1 -- 172.21.15.83:0/3320723215 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc005480 con 0x7faed00aa470 2023-10-06T13:52:23.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faecd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:23.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faecd7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:52:23.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faecd7fa640 10 monclient: dump: 2023-10-06T13:52:23.548 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:23.548 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.548 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.548 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.548 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:23.548 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.549 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:23.549 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:23.549 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:23.549 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faecd7fa640 10 monclient: _finish_auth 0 2023-10-06T13:52:23.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faecd7fa640 10 monclient: _check_auth_tickets 2023-10-06T13:52:23.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faecd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:23.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faecd7fa640 1 -- 172.21.15.83:0/3320723215 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc003610 con 0x7faed00aa470 2023-10-06T13:52:23.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faed7084640 5 monclient: authenticate success, global_id 4135 2023-10-06T13:52:23.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faed7084640 10 monclient: _renew_subs 2023-10-06T13:52:23.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faed7084640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:23.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faed7084640 1 -- 172.21.15.83:0/3320723215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed00aaff0 con 0x7faed00aa470 2023-10-06T13:52:23.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faecd7fa640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:23.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faecd7fa640 1 -- 172.21.15.83:0/3320723215 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc003980 con 0x7faed00aa470 2023-10-06T13:52:23.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faecd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:23.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faecd7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:23.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faecd7fa640 10 monclient: dump: 2023-10-06T13:52:23.550 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:23.550 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.550 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.550 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.550 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:23.550 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.550 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:23.550 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faeceffd640 4 set_mon_vals no callback set 2023-10-06T13:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faed7084640 10 client.4135.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faed7084640 10 monclient: _renew_subs 2023-10-06T13:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faed7084640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faed7084640 1 -- 172.21.15.83:0/3320723215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed01d94d0 con 0x7faed00aa470 2023-10-06T13:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.544+0000 7faed7084640 1 librados: init done 2023-10-06T13:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.545+0000 7faed7084640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.545+0000 7faed7084640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.545+0000 7faed7084640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.545+0000 7faed7084640 1 -- 172.21.15.83:0/3320723215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed00678e0 con 0x7faed00aa470 2023-10-06T13:52:23.563 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:23.561+0000 7f4569a0e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T13:52:23.579 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:23.577+0000 7f6f00a18040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T13:52:23.607 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:23.605+0000 7f4569a0e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T13:52:23.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.644+0000 7faecd7fa640 1 -- 172.21.15.83:0/3320723215 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faebc003ae0 con 0x7faed00aa470 2023-10-06T13:52:23.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.644+0000 7faecd7fa640 1 -- 172.21.15.83:0/3320723215 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7faebc01cb70 con 0x7faed00aa470 2023-10-06T13:52:23.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.644+0000 7faecd7fa640 10 client.4135.objecter ms_dispatch 0x7faed0000b90 osd_map(6..6 src has 1..6) v4 2023-10-06T13:52:23.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.644+0000 7faecd7fa640 3 client.4135.objecter handle_osd_map got epochs [6,6] > 0 2023-10-06T13:52:23.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.644+0000 7faecd7fa640 3 client.4135.objecter handle_osd_map decoding full epoch 6 2023-10-06T13:52:23.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.644+0000 7faecd7fa640 20 client.4135.objecter dump_active .. 0 homeless 2023-10-06T13:52:23.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.644+0000 7faecd7fa640 1 -- 172.21.15.83:0/3320723215 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7faebc01ce40 con 0x7faed00aa470 2023-10-06T13:52:23.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.644+0000 7faecd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:23.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.644+0000 7faecd7fa640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.716+0000 7faed7084640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "ad31d4df-e78c-4e32-bfd4-3fe6e70dd428", "id": 4}] 2023-10-06T13:52:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.716+0000 7faed7084640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "ad31d4df-e78c-4e32-bfd4-3fe6e70dd428", "id": 4}] 2023-10-06T13:52:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.716+0000 7faed7084640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.716+0000 7faed7084640 1 -- 172.21.15.83:0/3320723215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ad31d4df-e78c-4e32-bfd4-3fe6e70dd428", "id": 4} v 0) v1 -- 0x7faed00ad9f0 con 0x7faed00aa470 2023-10-06T13:52:23.755 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:23.753+0000 7f4569a0e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T13:52:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.783+0000 7faecd7fa640 1 -- 172.21.15.83:0/3320723215 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ad31d4df-e78c-4e32-bfd4-3fe6e70dd428", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7faebc01c460 con 0x7faed00aa470 2023-10-06T13:52:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.783+0000 7faecd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "ad31d4df-e78c-4e32-bfd4-3fe6e70dd428", "id": 4}] 2023-10-06T13:52:23.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.783+0000 7faecd7fa640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:52:23.785 INFO:teuthology.orchestra.run.smithi083.stdout:4 2023-10-06T13:52:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.783+0000 7faed7084640 10 librados: watch_flush enter 2023-10-06T13:52:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 10 librados: watch_flush exit 2023-10-06T13:52:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 20 client.4135.objecter shutdown clearing up homeless session... 2023-10-06T13:52:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 10 client.4135.objecter successfully canceled tick 2023-10-06T13:52:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 10 monclient: shutdown 2023-10-06T13:52:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 1 -- 172.21.15.83:0/3320723215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed00aa470 msgr2=0x7faed00a9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 1 --2- 172.21.15.83:0/3320723215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed00aa470 0x7faed00a9d50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faebc00d4f0 tx=0x7faebc00d9b0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 1 -- 172.21.15.83:0/3320723215 shutdown_connections 2023-10-06T13:52:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 1 --2- 172.21.15.83:0/3320723215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed01d5d10 0x7faed01d61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 1 --2- 172.21.15.83:0/3320723215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed00aa470 0x7faed00a9d50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 1 --2- 172.21.15.83:0/3320723215 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faec00051c0 0x7faed0144440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 1 -- 172.21.15.83:0/3320723215 >> 172.21.15.83:0/3320723215 conn(0x7faed00b6e00 msgr2=0x7faed00b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 1 -- 172.21.15.83:0/3320723215 shutdown_connections 2023-10-06T13:52:23.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 1 -- 172.21.15.83:0/3320723215 wait complete. 2023-10-06T13:52:23.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.784+0000 7faed7084640 1 librados: shutdown 2023-10-06T13:52:23.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new 931b23f8-80c3-4595-8931-2568be479cb6 5 2023-10-06T13:52:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.933+0000 7f5bf1877640 10 monclient: get_monmap_and_config 2023-10-06T13:52:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.933+0000 7f5bf1877640 10 monclient: build_initial_monmap 2023-10-06T13:52:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.933+0000 7f5bf1877640 10 monclient: monmap: 2023-10-06T13:52:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:52:23.934173+0000 2023-10-06T13:52:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:52:23.934173+0000 2023-10-06T13:52:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:52:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:52:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:52:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:52:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:52:23.936 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 1 Processor -- start 2023-10-06T13:52:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 1 -- start start 2023-10-06T13:52:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 10 monclient: init 2023-10-06T13:52:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 10 monclient: _add_conns ranks=[2,0,1,4,3] 2023-10-06T13:52:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bec06b6d0 0x7f5bec144200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 10 monclient(hunting): picked mon.noname-b con 0x7f5bec06b6d0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5bec144740 addr v1:172.21.15.19:6789/0 2023-10-06T13:52:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5bec13e630 addr v1:172.21.15.83:6789/0 2023-10-06T13:52:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bec06baa0 con 0x7f5bec144740 2023-10-06T13:52:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bec0b2530 con 0x7f5bec13e630 2023-10-06T13:52:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec13ea00 con 0x7f5bec06b6d0 2023-10-06T13:52:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.934+0000 7f5bf1877640 10 monclient(hunting): authenticate will time out at 1272.267169s 2023-10-06T13:52:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.935+0000 7f5beb7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5bec13e630 0x7f5bec14b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56692/0 (socket says 172.21.15.83:56692) 2023-10-06T13:52:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.935+0000 7f5beb7fe640 1 -- 172.21.15.83:0/832465867 learned_addr learned my addr 172.21.15.83:0/832465867 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-10-06T13:52:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.935+0000 7f5beaffd640 1 --2- 172.21.15.83:0/832465867 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bec06b6d0 0x7f5bec144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.936+0000 7f5beaffd640 10 monclient(hunting): get_auth_request con 0x7f5bec06b6d0 auth_method 0 2023-10-06T13:52:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.936+0000 7f5beaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.936+0000 7f5beaffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.936+0000 7f5beaffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.936+0000 7f5beaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:23.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.936+0000 7f5beaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:23.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.936+0000 7f5beaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5bea7fc640 1 -- 172.21.15.83:0/832465867 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361436779 0 0) 0x7f5bec0b2530 con 0x7f5bec13e630 2023-10-06T13:52:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5bea7fc640 10 monclient(hunting): _negotiate 2023-10-06T13:52:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5bea7fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5bea7fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5bea7fc640 10 monclient(hunting): my global_id is 4138 2023-10-06T13:52:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5bea7fc640 1 -- 172.21.15.83:0/832465867 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bd8003520 con 0x7f5bec13e630 2023-10-06T13:52:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5bea7fc640 1 -- 172.21.15.83:0/832465867 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013667029 0 0) 0x7f5bec06baa0 con 0x7f5bec144740 2023-10-06T13:52:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5bea7fc640 10 monclient(hunting): _negotiate 2023-10-06T13:52:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5bea7fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5bea7fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5bea7fc640 10 monclient(hunting): my global_id is 4143 2023-10-06T13:52:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5bea7fc640 1 -- 172.21.15.83:0/832465867 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bec0b2530 con 0x7f5bec144740 2023-10-06T13:52:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5beaffd640 10 monclient(hunting): handle_auth_done global_id 4130 payload 274 2023-10-06T13:52:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5beaffd640 1 -- 172.21.15.83:0/832465867 >> v1:172.21.15.83:6789/0 conn(0x7f5bec13e630 legacy=0x7f5bec14b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5beaffd640 1 -- 172.21.15.83:0/832465867 >> v1:172.21.15.19:6789/0 conn(0x7f5bec144740 legacy=0x7f5bec13df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5beaffd640 10 monclient: _finish_hunting 0 2023-10-06T13:52:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5beaffd640 1 monclient: found mon.noname-b 2023-10-06T13:52:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5beaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5beaffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T13:52:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.938+0000 7f5beaffd640 1 -- 172.21.15.83:0/832465867 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec0b2810 con 0x7f5bec06b6d0 2023-10-06T13:52:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.940+0000 7f5beaffd640 1 --2- 172.21.15.83:0/832465867 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bec06b6d0 0x7f5bec144200 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0097d0 tx=0x7f5bdc02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5e28f9ba2fd70eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bea7fc640 1 -- 172.21.15.83:0/832465867 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc031890 con 0x7f5bec06b6d0 2023-10-06T13:52:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bea7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:52:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bea7fc640 10 monclient: dump: 2023-10-06T13:52:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.946 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:23.947 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bea7fc640 10 monclient: _finish_auth 0 2023-10-06T13:52:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bea7fc640 10 monclient: _check_auth_tickets 2023-10-06T13:52:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bea7fc640 1 -- 172.21.15.83:0/832465867 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc0319f0 con 0x7f5bec06b6d0 2023-10-06T13:52:23.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bf1877640 5 monclient: authenticate success, global_id 4130 2023-10-06T13:52:23.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bf1877640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:52:23.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bea7fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:23.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bf1877640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:23.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bf1877640 4 set_mon_vals no callback set 2023-10-06T13:52:23.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bea7fc640 1 -- 172.21.15.83:0/832465867 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc031d80 con 0x7f5bec06b6d0 2023-10-06T13:52:23.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:23.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bea7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:52:23.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.944+0000 7f5bea7fc640 10 monclient: dump: 2023-10-06T13:52:23.948 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:23.948 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.948 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.948 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:23.949 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.945+0000 7f5bf1877640 10 monclient: shutdown 2023-10-06T13:52:23.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.945+0000 7f5bf1877640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:23.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.945+0000 7f5bf1877640 1 -- 172.21.15.83:0/832465867 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bec06b6d0 msgr2=0x7f5bec144200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.945+0000 7f5bf1877640 1 --2- 172.21.15.83:0/832465867 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bec06b6d0 0x7f5bec144200 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0097d0 tx=0x7f5bdc02b930 comp rx=0 tx=0).stop 2023-10-06T13:52:23.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.945+0000 7f5bf1877640 1 -- 172.21.15.83:0/832465867 shutdown_connections 2023-10-06T13:52:23.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.945+0000 7f5bf1877640 1 --2- 172.21.15.83:0/832465867 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bec06b6d0 0x7f5bec144200 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.945+0000 7f5bf1877640 1 -- 172.21.15.83:0/832465867 >> 172.21.15.83:0/832465867 conn(0x7f5bec0665b0 msgr2=0x7f5bec06ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.945+0000 7f5bf1877640 1 -- 172.21.15.83:0/832465867 shutdown_connections 2023-10-06T13:52:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.945+0000 7f5bf1877640 1 -- 172.21.15.83:0/832465867 wait complete. 2023-10-06T13:52:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.946+0000 7f5bf1877640 10 monclient: build_initial_monmap 2023-10-06T13:52:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.946+0000 7f5bf1877640 10 monclient: monmap: 2023-10-06T13:52:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 0.000000 2023-10-06T13:52:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:created 0.000000 2023-10-06T13:52:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.955 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:23.955 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:23.955 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:23.955 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.946+0000 7f5bf1877640 1 librados: starting msgr at 2023-10-06T13:52:23.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.946+0000 7f5bf1877640 1 librados: starting objecter 2023-10-06T13:52:23.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.946+0000 7f5bf1877640 1 Processor -- start 2023-10-06T13:52:23.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.946+0000 7f5bf1877640 1 -- start start 2023-10-06T13:52:23.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.946+0000 7f5bf1877640 1 librados: setting wanted keys 2023-10-06T13:52:23.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.946+0000 7f5bf1877640 1 librados: calling monclient init 2023-10-06T13:52:23.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.946+0000 7f5bf1877640 10 monclient: init 2023-10-06T13:52:23.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:23.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:52:23.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bec06b6d0 0x7f5bec1d5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 10 monclient(hunting): picked mon.noname-a con 0x7f5bec06b6d0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:23.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bec13e630 0x7f5bec13ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 10 monclient(hunting): picked mon.noname-c con 0x7f5bec13e630 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:23.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec144740 0x7f5bec13b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:23.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 10 monclient(hunting): picked mon.noname-b con 0x7f5bec144740 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:23.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0aa730 con 0x7f5bec06b6d0 2023-10-06T13:52:23.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0aa940 con 0x7f5bec144740 2023-10-06T13:52:23.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:23.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec13b5b0 con 0x7f5bec13e630 2023-10-06T13:52:23.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:23.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5bf1877640 10 monclient(hunting): authenticate will time out at 1272.279567s 2023-10-06T13:52:23.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5beb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec144740 0x7f5bec13b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:23.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5beaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bec06b6d0 0x7f5bec1d5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:23.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.947+0000 7f5be3fff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bec13e630 0x7f5bec13ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:23.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec144740 0x7f5bec13b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48372/0 (socket says 172.21.15.83:48372) 2023-10-06T13:52:23.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beb7fe640 1 -- 172.21.15.83:0/4070120086 learned_addr learned my addr 172.21.15.83:0/4070120086 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-10-06T13:52:23.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beb7fe640 10 monclient(hunting): get_auth_request con 0x7f5bec144740 auth_method 0 2023-10-06T13:52:23.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:23.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beb7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beb7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beaffd640 10 monclient(hunting): get_auth_request con 0x7f5bec06b6d0 auth_method 0 2023-10-06T13:52:23.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:23.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beaffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beaffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5be3fff640 10 monclient(hunting): get_auth_request con 0x7f5bec13e630 auth_method 0 2023-10-06T13:52:23.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5be3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:23.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5be3fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:23.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5be3fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:23.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:23.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:23.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:23.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:23.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:23.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5beaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:23.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5be3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:23.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5be3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:23.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.948+0000 7f5be3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:23.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.951+0000 7f5beb7fe640 10 monclient(hunting): handle_auth_done global_id 4141 payload 824 2023-10-06T13:52:23.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.951+0000 7f5beb7fe640 1 -- 172.21.15.83:0/4070120086 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bec13e630 msgr2=0x7f5bec13ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.951+0000 7f5beb7fe640 1 --2- 172.21.15.83:0/4070120086 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bec13e630 0x7f5bec13ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.951+0000 7f5beb7fe640 1 -- 172.21.15.83:0/4070120086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bec06b6d0 msgr2=0x7f5bec1d5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:23.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.951+0000 7f5beb7fe640 1 --2- 172.21.15.83:0/4070120086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bec06b6d0 0x7f5bec1d5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:23.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.951+0000 7f5beb7fe640 10 monclient: _finish_hunting 0 2023-10-06T13:52:23.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.951+0000 7f5beb7fe640 1 monclient: found mon.noname-b 2023-10-06T13:52:23.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.951+0000 7f5beb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:23.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.951+0000 7f5beb7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:52:23.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.951+0000 7f5beb7fe640 1 -- 172.21.15.83:0/4070120086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc009400 con 0x7f5bec144740 2023-10-06T13:52:23.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.954+0000 7f5beb7fe640 1 --2- 172.21.15.83:0/4070120086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec144740 0x7f5bec13b070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5be400cb40 tx=0x7f5be400a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:23.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.954+0000 7f5be8ff9640 10 client.?.objecter ms_handle_connect 0x7f5bec144740 2023-10-06T13:52:23.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.954+0000 7f5be8ff9640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:23.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5be8ff9640 1 -- 172.21.15.83:0/4070120086 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be400aca0 con 0x7f5bec144740 2023-10-06T13:52:23.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5be8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:23.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5be8ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:52:23.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5be8ff9640 10 monclient: dump: 2023-10-06T13:52:23.962 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:23.962 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.962 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.962 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.962 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:23.962 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.962 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:23.963 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:23.963 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:23.963 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5be8ff9640 10 monclient: _finish_auth 0 2023-10-06T13:52:23.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5be8ff9640 10 monclient: _check_auth_tickets 2023-10-06T13:52:23.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5be8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:23.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5be8ff9640 1 -- 172.21.15.83:0/4070120086 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be400ae20 con 0x7f5bec144740 2023-10-06T13:52:23.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5be8ff9640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:23.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5be8ff9640 1 -- 172.21.15.83:0/4070120086 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4012640 con 0x7f5bec144740 2023-10-06T13:52:23.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5be8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:23.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5be8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:23.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5be8ff9640 10 monclient: dump: 2023-10-06T13:52:23.964 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:23.964 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:23.964 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.964 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:23.964 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:23.964 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:23.964 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:23.964 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:23.964 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:23.964 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:23.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5bf1877640 5 monclient: authenticate success, global_id 4141 2023-10-06T13:52:23.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5bf1877640 10 monclient: _renew_subs 2023-10-06T13:52:23.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5bf1877640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:23.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5bf1877640 1 -- 172.21.15.83:0/4070120086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bec13b830 con 0x7f5bec144740 2023-10-06T13:52:23.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5be9ffb640 4 set_mon_vals no callback set 2023-10-06T13:52:23.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5bf1877640 10 client.4141.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:23.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5bf1877640 10 monclient: _renew_subs 2023-10-06T13:52:23.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5bf1877640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:23.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5bf1877640 1 -- 172.21.15.83:0/4070120086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec1d64e0 con 0x7f5bec144740 2023-10-06T13:52:23.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.957+0000 7f5bf1877640 1 librados: init done 2023-10-06T13:52:23.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.958+0000 7f5bf1877640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:23.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.958+0000 7f5bf1877640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:23.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.958+0000 7f5bf1877640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:23.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:23.958+0000 7f5bf1877640 1 -- 172.21.15.83:0/4070120086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bec04e000 con 0x7f5bec144740 2023-10-06T13:52:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.059+0000 7f5be8ff9640 1 -- 172.21.15.83:0/4070120086 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5be4012aa0 con 0x7f5bec144740 2023-10-06T13:52:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.059+0000 7f5be8ff9640 1 -- 172.21.15.83:0/4070120086 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f5be40075f0 con 0x7f5bec144740 2023-10-06T13:52:24.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.059+0000 7f5be8ff9640 10 client.4141.objecter ms_dispatch 0x7f5bec000b90 osd_map(7..7 src has 1..7) v4 2023-10-06T13:52:24.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.059+0000 7f5be8ff9640 3 client.4141.objecter handle_osd_map got epochs [7,7] > 0 2023-10-06T13:52:24.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.059+0000 7f5be8ff9640 3 client.4141.objecter handle_osd_map decoding full epoch 7 2023-10-06T13:52:24.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.059+0000 7f5be8ff9640 20 client.4141.objecter dump_active .. 0 homeless 2023-10-06T13:52:24.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.059+0000 7f5be8ff9640 1 -- 172.21.15.83:0/4070120086 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f5be4017070 con 0x7f5bec144740 2023-10-06T13:52:24.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.059+0000 7f5be8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:24.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.059+0000 7f5be8ff9640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.130+0000 7f5bf1877640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "931b23f8-80c3-4595-8931-2568be479cb6", "id": 5}] 2023-10-06T13:52:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.130+0000 7f5bf1877640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "931b23f8-80c3-4595-8931-2568be479cb6", "id": 5}] 2023-10-06T13:52:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.130+0000 7f5bf1877640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.130+0000 7f5bf1877640 1 -- 172.21.15.83:0/4070120086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd new", "uuid": "931b23f8-80c3-4595-8931-2568be479cb6", "id": 5} v 0) v1 -- 0x7f5bec0aab00 con 0x7f5bec144740 2023-10-06T13:52:24.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.225+0000 7f5be8ff9640 1 -- 172.21.15.83:0/4070120086 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "931b23f8-80c3-4595-8931-2568be479cb6", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5be4007840 con 0x7f5bec144740 2023-10-06T13:52:24.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.225+0000 7f5be8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "931b23f8-80c3-4595-8931-2568be479cb6", "id": 5}] 2023-10-06T13:52:24.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.225+0000 7f5be8ff9640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:52:24.227 INFO:teuthology.orchestra.run.smithi083.stdout:5 2023-10-06T13:52:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 10 librados: watch_flush enter 2023-10-06T13:52:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 10 librados: watch_flush exit 2023-10-06T13:52:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 20 client.4141.objecter shutdown clearing up homeless session... 2023-10-06T13:52:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 10 client.4141.objecter successfully canceled tick 2023-10-06T13:52:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 10 monclient: shutdown 2023-10-06T13:52:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 1 -- 172.21.15.83:0/4070120086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec144740 msgr2=0x7f5bec13b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 1 --2- 172.21.15.83:0/4070120086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec144740 0x7f5bec13b070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5be400cb40 tx=0x7f5be400a060 comp rx=0 tx=0).stop 2023-10-06T13:52:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 1 -- 172.21.15.83:0/4070120086 shutdown_connections 2023-10-06T13:52:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 1 --2- 172.21.15.83:0/4070120086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec144740 0x7f5bec13b070 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 1 --2- 172.21.15.83:0/4070120086 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bec13e630 0x7f5bec13ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 1 --2- 172.21.15.83:0/4070120086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bec06b6d0 0x7f5bec1d5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 1 -- 172.21.15.83:0/4070120086 >> 172.21.15.83:0/4070120086 conn(0x7f5bec0665b0 msgr2=0x7f5bec13be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 1 -- 172.21.15.83:0/4070120086 shutdown_connections 2023-10-06T13:52:24.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 1 -- 172.21.15.83:0/4070120086 wait complete. 2023-10-06T13:52:24.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.226+0000 7f5bf1877640 1 librados: shutdown 2023-10-06T13:52:24.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new 70dd2f95-93eb-448a-a65d-3a09bdea85a1 6 2023-10-06T13:52:24.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.336+0000 7fb4206f1640 10 monclient: get_monmap_and_config 2023-10-06T13:52:24.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.336+0000 7fb4206f1640 10 monclient: build_initial_monmap 2023-10-06T13:52:24.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.336+0000 7fb4206f1640 10 monclient: monmap: 2023-10-06T13:52:24.338 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:24.339 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:24.339 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:52:24.337221+0000 2023-10-06T13:52:24.339 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:52:24.337221+0000 2023-10-06T13:52:24.339 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:24.339 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:24.339 INFO:teuthology.orchestra.run.smithi083.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:52:24.339 INFO:teuthology.orchestra.run.smithi083.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:52:24.339 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:52:24.339 INFO:teuthology.orchestra.run.smithi083.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:52:24.339 INFO:teuthology.orchestra.run.smithi083.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:52:24.340 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:24.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 1 Processor -- start 2023-10-06T13:52:24.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 1 -- start start 2023-10-06T13:52:24.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 10 monclient: init 2023-10-06T13:52:24.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:24.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 10 monclient: _add_conns ranks=[3,0,1,2,4] 2023-10-06T13:52:24.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb4180b3990 0x7fb4180af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:24.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 10 monclient(hunting): picked mon.noname-a con 0x7fb4180b3990 addr v2:172.21.15.19:3300/0 2023-10-06T13:52:24.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb4180b3d60 addr v1:172.21.15.19:6789/0 2023-10-06T13:52:24.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb4180b00c0 addr v1:172.21.15.83:6789/0 2023-10-06T13:52:24.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4180aebe0 con 0x7fb4180b3d60 2023-10-06T13:52:24.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb418141070 con 0x7fb4180b00c0 2023-10-06T13:52:24.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:24.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb418140dd0 con 0x7fb4180b3990 2023-10-06T13:52:24.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:24.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.337+0000 7fb4206f1640 10 monclient(hunting): authenticate will time out at 1272.670223s 2023-10-06T13:52:24.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.338+0000 7fb41ec8e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb4180b00c0 0x7fb41814b600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56694/0 (socket says 172.21.15.83:56694) 2023-10-06T13:52:24.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.338+0000 7fb41ec8e640 1 -- 172.21.15.83:0/2101555423 learned_addr learned my addr 172.21.15.83:0/2101555423 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-10-06T13:52:24.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.338+0000 7fb41e48d640 1 --2- 172.21.15.83:0/2101555423 >> v2:172.21.15.19:3300/0 conn(0x7fb4180b3990 0x7fb4180af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:24.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.339+0000 7fb41e48d640 10 monclient(hunting): get_auth_request con 0x7fb4180b3990 auth_method 0 2023-10-06T13:52:24.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.339+0000 7fb41e48d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:24.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.339+0000 7fb41e48d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:24.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.339+0000 7fb41e48d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:24.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.339+0000 7fb41e48d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:24.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.339+0000 7fb41e48d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:24.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.339+0000 7fb41e48d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:24.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.341+0000 7fb41d48b640 1 -- 172.21.15.83:0/2101555423 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828991028 0 0) 0x7fb418141070 con 0x7fb4180b00c0 2023-10-06T13:52:24.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.341+0000 7fb41d48b640 10 monclient(hunting): _negotiate 2023-10-06T13:52:24.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.341+0000 7fb41d48b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:24.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.341+0000 7fb41d48b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:24.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.341+0000 7fb41d48b640 10 monclient(hunting): my global_id is 4144 2023-10-06T13:52:24.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.341+0000 7fb41d48b640 1 -- 172.21.15.83:0/2101555423 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb404003520 con 0x7fb4180b00c0 2023-10-06T13:52:24.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.341+0000 7fb41e48d640 10 monclient(hunting): handle_auth_done global_id 4149 payload 274 2023-10-06T13:52:24.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.341+0000 7fb41e48d640 1 -- 172.21.15.83:0/2101555423 >> v1:172.21.15.83:6789/0 conn(0x7fb4180b00c0 legacy=0x7fb41814b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:24.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.341+0000 7fb41e48d640 1 -- 172.21.15.83:0/2101555423 >> v1:172.21.15.19:6789/0 conn(0x7fb4180b3d60 legacy=0x7fb418141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:24.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.341+0000 7fb41e48d640 10 monclient: _finish_hunting 0 2023-10-06T13:52:24.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.341+0000 7fb41e48d640 1 monclient: found mon.noname-a 2023-10-06T13:52:24.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.341+0000 7fb41e48d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:24.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.341+0000 7fb41e48d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:52:24.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.341+0000 7fb41e48d640 1 -- 172.21.15.83:0/2101555423 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb418141c10 con 0x7fb4180b3990 2023-10-06T13:52:24.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.345+0000 7fb41e48d640 1 --2- 172.21.15.83:0/2101555423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4180b3990 0x7fb4180af810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb4140097a0 tx=0x7fb41402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872916e8989beee4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:24.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.347+0000 7fb41d48b640 1 -- 172.21.15.83:0/2101555423 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb414031830 con 0x7fb4180b3990 2023-10-06T13:52:24.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.347+0000 7fb41d48b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:24.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.347+0000 7fb41d48b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:52:24.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.347+0000 7fb41d48b640 10 monclient: dump: 2023-10-06T13:52:24.349 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:24.349 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:24.349 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.349 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.349 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:24.349 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:24.349 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:24.349 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:24.349 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:24.349 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:24.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.347+0000 7fb41d48b640 10 monclient: _finish_auth 0 2023-10-06T13:52:24.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.347+0000 7fb41d48b640 10 monclient: _check_auth_tickets 2023-10-06T13:52:24.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.347+0000 7fb41d48b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:24.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.347+0000 7fb4206f1640 5 monclient: authenticate success, global_id 4149 2023-10-06T13:52:24.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.347+0000 7fb4206f1640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:52:24.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb41d48b640 1 -- 172.21.15.83:0/2101555423 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb414031990 con 0x7fb4180b3990 2023-10-06T13:52:24.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb41d48b640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:24.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb41d48b640 1 -- 172.21.15.83:0/2101555423 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb414031d00 con 0x7fb4180b3990 2023-10-06T13:52:24.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb4206f1640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:24.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb41d48b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:24.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb4206f1640 4 set_mon_vals no callback set 2023-10-06T13:52:24.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb41d48b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:52:24.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb41d48b640 10 monclient: dump: 2023-10-06T13:52:24.352 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:24.352 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:24.352 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.353 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.353 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:24.353 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:24.353 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:24.353 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:24.353 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:24.353 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:24.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb4206f1640 10 monclient: shutdown 2023-10-06T13:52:24.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb4206f1640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:24.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb4206f1640 1 -- 172.21.15.83:0/2101555423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4180b3990 msgr2=0x7fb4180af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:24.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb4206f1640 1 --2- 172.21.15.83:0/2101555423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4180b3990 0x7fb4180af810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb4140097a0 tx=0x7fb41402b8d0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb4206f1640 1 -- 172.21.15.83:0/2101555423 shutdown_connections 2023-10-06T13:52:24.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb4206f1640 1 --2- 172.21.15.83:0/2101555423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4180b3990 0x7fb4180af810 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb4206f1640 1 -- 172.21.15.83:0/2101555423 >> 172.21.15.83:0/2101555423 conn(0x7fb4180b6e00 msgr2=0x7fb4180b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:24.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb4206f1640 1 -- 172.21.15.83:0/2101555423 shutdown_connections 2023-10-06T13:52:24.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.349+0000 7fb4206f1640 1 -- 172.21.15.83:0/2101555423 wait complete. 2023-10-06T13:52:24.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.350+0000 7fb4206f1640 10 monclient: build_initial_monmap 2023-10-06T13:52:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.350+0000 7fb4206f1640 10 monclient: monmap: 2023-10-06T13:52:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 0.000000 2023-10-06T13:52:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:created 0.000000 2023-10-06T13:52:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:24.357 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:24.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.350+0000 7fb4206f1640 1 librados: starting msgr at 2023-10-06T13:52:24.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.350+0000 7fb4206f1640 1 librados: starting objecter 2023-10-06T13:52:24.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.350+0000 7fb4206f1640 1 Processor -- start 2023-10-06T13:52:24.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.350+0000 7fb4206f1640 1 -- start start 2023-10-06T13:52:24.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.350+0000 7fb4206f1640 1 librados: setting wanted keys 2023-10-06T13:52:24.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.350+0000 7fb4206f1640 1 librados: calling monclient init 2023-10-06T13:52:24.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.350+0000 7fb4206f1640 10 monclient: init 2023-10-06T13:52:24.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:24.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:52:24.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4180b00c0 0x7fb418144210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:24.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 10 monclient(hunting): picked mon.noname-a con 0x7fb4180b00c0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:24.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb4180b3990 0x7fb4180accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:24.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 10 monclient(hunting): picked mon.noname-c con 0x7fb4180b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:24.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4180b3d60 0x7fb4181d9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:24.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 10 monclient(hunting): picked mon.noname-b con 0x7fb4180b3d60 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:24.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:24.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4181d9680 con 0x7fb4180b00c0 2023-10-06T13:52:24.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:24.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4181d6100 con 0x7fb4180b3d60 2023-10-06T13:52:24.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:24.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180ad210 con 0x7fb4180b3990 2023-10-06T13:52:24.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:24.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb4206f1640 10 monclient(hunting): authenticate will time out at 1272.683732s 2023-10-06T13:52:24.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.351+0000 7fb41ec8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4180b3d60 0x7fb4181d9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:24.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41e48d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4180b00c0 0x7fb418144210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:24.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41dc8c640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb4180b3990 0x7fb4180accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:24.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41ec8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4180b3d60 0x7fb4181d9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48374/0 (socket says 172.21.15.83:48374) 2023-10-06T13:52:24.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41ec8e640 1 -- 172.21.15.83:0/418707340 learned_addr learned my addr 172.21.15.83:0/418707340 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-10-06T13:52:24.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41ec8e640 10 monclient(hunting): get_auth_request con 0x7fb4180b3d60 auth_method 0 2023-10-06T13:52:24.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41ec8e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:24.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41ec8e640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:24.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41ec8e640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:24.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41e48d640 10 monclient(hunting): get_auth_request con 0x7fb4180b00c0 auth_method 0 2023-10-06T13:52:24.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41e48d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:24.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41e48d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:24.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41e48d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:24.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41dc8c640 10 monclient(hunting): get_auth_request con 0x7fb4180b3990 auth_method 0 2023-10-06T13:52:24.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41dc8c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:24.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41dc8c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:24.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41dc8c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:24.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41ec8e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:24.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41ec8e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:24.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41ec8e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:24.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41e48d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:24.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41e48d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:24.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.352+0000 7fb41e48d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:24.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.353+0000 7fb41dc8c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:24.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.353+0000 7fb41dc8c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:24.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.353+0000 7fb41dc8c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:24.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.355+0000 7fb41ec8e640 10 monclient(hunting): handle_auth_done global_id 4147 payload 824 2023-10-06T13:52:24.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.355+0000 7fb41ec8e640 1 -- 172.21.15.83:0/418707340 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb4180b3990 msgr2=0x7fb4180accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:24.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.355+0000 7fb41ec8e640 1 --2- 172.21.15.83:0/418707340 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb4180b3990 0x7fb4180accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.355+0000 7fb41ec8e640 1 -- 172.21.15.83:0/418707340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4180b00c0 msgr2=0x7fb418144210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:24.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.355+0000 7fb41ec8e640 1 --2- 172.21.15.83:0/418707340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4180b00c0 0x7fb418144210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.355+0000 7fb41ec8e640 10 monclient: _finish_hunting 0 2023-10-06T13:52:24.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.355+0000 7fb41ec8e640 1 monclient: found mon.noname-b 2023-10-06T13:52:24.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.355+0000 7fb41ec8e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:24.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.355+0000 7fb41ec8e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:52:24.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.355+0000 7fb41ec8e640 1 -- 172.21.15.83:0/418707340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb414009400 con 0x7fb4180b3d60 2023-10-06T13:52:24.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.358+0000 7fb41ec8e640 1 --2- 172.21.15.83:0/418707340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4180b3d60 0x7fb4181d9b10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb41000cb40 tx=0x7fb41000a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:24.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.358+0000 7fb40effd640 10 client.?.objecter ms_handle_connect 0x7fb4180b3d60 2023-10-06T13:52:24.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.358+0000 7fb40effd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:24.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb40effd640 1 -- 172.21.15.83:0/418707340 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41000aca0 con 0x7fb4180b3d60 2023-10-06T13:52:24.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb40effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:24.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb40effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:52:24.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb40effd640 10 monclient: dump: 2023-10-06T13:52:24.365 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:24.365 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:24.365 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.365 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.365 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:24.365 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:24.365 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:24.365 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:24.365 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:24.365 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:24.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb40effd640 10 monclient: _finish_auth 0 2023-10-06T13:52:24.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb40effd640 10 monclient: _check_auth_tickets 2023-10-06T13:52:24.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb40effd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:24.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb40effd640 1 -- 172.21.15.83:0/418707340 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41000ae00 con 0x7fb4180b3d60 2023-10-06T13:52:24.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb4206f1640 5 monclient: authenticate success, global_id 4147 2023-10-06T13:52:24.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb4206f1640 10 monclient: _renew_subs 2023-10-06T13:52:24.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb4206f1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:24.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb4206f1640 1 -- 172.21.15.83:0/418707340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4180ab060 con 0x7fb4180b3d60 2023-10-06T13:52:24.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb40effd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:24.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb40effd640 1 -- 172.21.15.83:0/418707340 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410012620 con 0x7fb4180b3d60 2023-10-06T13:52:24.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb40effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:24.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb40effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb40effd640 10 monclient: dump: 2023-10-06T13:52:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:24.367 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb41cc8a640 4 set_mon_vals no callback set 2023-10-06T13:52:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb4206f1640 10 client.4147.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb4206f1640 10 monclient: _renew_subs 2023-10-06T13:52:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb4206f1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb4206f1640 1 -- 172.21.15.83:0/418707340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4181da230 con 0x7fb4180b3d60 2023-10-06T13:52:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.362+0000 7fb4206f1640 1 librados: init done 2023-10-06T13:52:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.363+0000 7fb4206f1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.363+0000 7fb4206f1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.363+0000 7fb4206f1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.363+0000 7fb4206f1640 1 -- 172.21.15.83:0/418707340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4181d97f0 con 0x7fb4180b3d60 2023-10-06T13:52:24.426 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:24.425+0000 7f6f00a18040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T13:52:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.461+0000 7fb40effd640 1 -- 172.21.15.83:0/418707340 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb410012a80 con 0x7fb4180b3d60 2023-10-06T13:52:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.461+0000 7fb40effd640 1 -- 172.21.15.83:0/418707340 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fb410007660 con 0x7fb4180b3d60 2023-10-06T13:52:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.461+0000 7fb40effd640 10 client.4147.objecter ms_dispatch 0x7fb418000b90 osd_map(8..8 src has 1..8) v4 2023-10-06T13:52:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.461+0000 7fb40effd640 3 client.4147.objecter handle_osd_map got epochs [8,8] > 0 2023-10-06T13:52:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.461+0000 7fb40effd640 3 client.4147.objecter handle_osd_map decoding full epoch 8 2023-10-06T13:52:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.461+0000 7fb40effd640 20 client.4147.objecter dump_active .. 0 homeless 2023-10-06T13:52:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.461+0000 7fb40effd640 1 -- 172.21.15.83:0/418707340 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fb410017070 con 0x7fb4180b3d60 2023-10-06T13:52:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.461+0000 7fb40effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.461+0000 7fb40effd640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.533+0000 7fb4206f1640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "70dd2f95-93eb-448a-a65d-3a09bdea85a1", "id": 6}] 2023-10-06T13:52:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.533+0000 7fb4206f1640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "70dd2f95-93eb-448a-a65d-3a09bdea85a1", "id": 6}] 2023-10-06T13:52:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.533+0000 7fb4206f1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.533+0000 7fb4206f1640 1 -- 172.21.15.83:0/418707340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd new", "uuid": "70dd2f95-93eb-448a-a65d-3a09bdea85a1", "id": 6} v 0) v1 -- 0x7fb4180678e0 con 0x7fb4180b3d60 2023-10-06T13:52:24.595 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:24.593+0000 7f4569a0e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T13:52:24.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.597+0000 7fb40effd640 1 -- 172.21.15.83:0/418707340 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "70dd2f95-93eb-448a-a65d-3a09bdea85a1", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb410007a40 con 0x7fb4180b3d60 2023-10-06T13:52:24.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.597+0000 7fb40effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "70dd2f95-93eb-448a-a65d-3a09bdea85a1", "id": 6}] 2023-10-06T13:52:24.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.597+0000 7fb40effd640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:52:24.599 INFO:teuthology.orchestra.run.smithi083.stdout:6 2023-10-06T13:52:24.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.598+0000 7fb4206f1640 10 librados: watch_flush enter 2023-10-06T13:52:24.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.598+0000 7fb4206f1640 10 librados: watch_flush exit 2023-10-06T13:52:24.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.598+0000 7fb4206f1640 20 client.4147.objecter shutdown clearing up homeless session... 2023-10-06T13:52:24.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.598+0000 7fb4206f1640 10 client.4147.objecter successfully canceled tick 2023-10-06T13:52:24.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.598+0000 7fb4206f1640 10 monclient: shutdown 2023-10-06T13:52:24.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.598+0000 7fb4206f1640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:24.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.598+0000 7fb4206f1640 1 -- 172.21.15.83:0/418707340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4180b3d60 msgr2=0x7fb4181d9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:24.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.598+0000 7fb4206f1640 1 --2- 172.21.15.83:0/418707340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4180b3d60 0x7fb4181d9b10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb41000cb40 tx=0x7fb41000a060 comp rx=0 tx=0).stop 2023-10-06T13:52:24.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.598+0000 7fb4206f1640 1 -- 172.21.15.83:0/418707340 shutdown_connections 2023-10-06T13:52:24.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.598+0000 7fb4206f1640 1 --2- 172.21.15.83:0/418707340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4180b3d60 0x7fb4181d9b10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.598+0000 7fb4206f1640 1 --2- 172.21.15.83:0/418707340 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb4180b3990 0x7fb4180accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.598+0000 7fb4206f1640 1 --2- 172.21.15.83:0/418707340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4180b00c0 0x7fb418144210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.598+0000 7fb4206f1640 1 -- 172.21.15.83:0/418707340 >> 172.21.15.83:0/418707340 conn(0x7fb4180b6e00 msgr2=0x7fb4180b4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:24.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.599+0000 7fb4206f1640 1 -- 172.21.15.83:0/418707340 shutdown_connections 2023-10-06T13:52:24.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.599+0000 7fb4206f1640 1 -- 172.21.15.83:0/418707340 wait complete. 2023-10-06T13:52:24.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.599+0000 7fb4206f1640 1 librados: shutdown 2023-10-06T13:52:24.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new 9797eb84-1b2c-4280-ab13-bdb7b2007dd8 7 2023-10-06T13:52:24.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.710+0000 7f739240b640 10 monclient: get_monmap_and_config 2023-10-06T13:52:24.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.710+0000 7f739240b640 10 monclient: build_initial_monmap 2023-10-06T13:52:24.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.710+0000 7f739240b640 10 monclient: monmap: 2023-10-06T13:52:24.712 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:24.712 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:24.712 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:52:24.711350+0000 2023-10-06T13:52:24.713 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:52:24.711350+0000 2023-10-06T13:52:24.713 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:24.713 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:24.713 INFO:teuthology.orchestra.run.smithi083.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:52:24.713 INFO:teuthology.orchestra.run.smithi083.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:52:24.713 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:52:24.713 INFO:teuthology.orchestra.run.smithi083.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:52:24.713 INFO:teuthology.orchestra.run.smithi083.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:52:24.713 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:24.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.711+0000 7f739240b640 1 Processor -- start 2023-10-06T13:52:24.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.711+0000 7f739240b640 1 -- start start 2023-10-06T13:52:24.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.711+0000 7f739240b640 10 monclient: init 2023-10-06T13:52:24.714 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:24.711+0000 7f6f00a18040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T13:52:24.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f739240b640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:24.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f739240b640 10 monclient: _add_conns ranks=[3,4,0,1,2] 2023-10-06T13:52:24.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f739240b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f738c0b3990 0x7f738c0af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:24.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f739240b640 10 monclient(hunting): picked mon.noname-a con 0x7f738c0b3990 addr v2:172.21.15.19:3300/0 2023-10-06T13:52:24.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f739240b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f738c0b3d60 0x7f738c141500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:24.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f739240b640 10 monclient(hunting): picked mon.noname-c con 0x7f738c0b3d60 addr v2:172.21.15.83:3300/0 2023-10-06T13:52:24.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f739240b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f738c141a40 addr v1:172.21.15.19:6789/0 2023-10-06T13:52:24.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f739240b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738c0aebe0 con 0x7f738c141a40 2023-10-06T13:52:24.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f739240b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:24.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f739240b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738c140dd0 con 0x7f738c0b3990 2023-10-06T13:52:24.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f739240b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:24.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f739240b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738c141070 con 0x7f738c0b3d60 2023-10-06T13:52:24.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f739240b640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:24.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f739240b640 10 monclient(hunting): authenticate will time out at 1273.044343s 2023-10-06T13:52:24.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f738b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f738c0b3d60 0x7f738c141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:24.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.712+0000 7f738bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f738c0b3990 0x7f738c0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:24.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.713+0000 7f738b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f738c0b3d60 0x7f738c141500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52340/0 (socket says 172.21.15.83:52340) 2023-10-06T13:52:24.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.713+0000 7f738b7fe640 1 -- 172.21.15.83:0/850350630 learned_addr learned my addr 172.21.15.83:0/850350630 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-10-06T13:52:24.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.713+0000 7f738b7fe640 10 monclient(hunting): get_auth_request con 0x7f738c0b3d60 auth_method 0 2023-10-06T13:52:24.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.713+0000 7f738b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:24.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.713+0000 7f738b7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:24.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.713+0000 7f738b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:24.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.713+0000 7f738bfff640 10 monclient(hunting): get_auth_request con 0x7f738c0b3990 auth_method 0 2023-10-06T13:52:24.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.713+0000 7f738bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:24.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.713+0000 7f738bfff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:24.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.713+0000 7f738bfff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:24.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.713+0000 7f738b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:24.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.713+0000 7f738b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:24.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.713+0000 7f738b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:24.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.714+0000 7f738bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:24.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.714+0000 7f738bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:24.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.714+0000 7f738bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:24.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.715+0000 7f738b7fe640 10 monclient(hunting): handle_auth_done global_id 4150 payload 274 2023-10-06T13:52:24.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.715+0000 7f738b7fe640 1 -- 172.21.15.83:0/850350630 >> v2:172.21.15.19:3300/0 conn(0x7f738c0b3990 msgr2=0x7f738c0af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:24.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.715+0000 7f738b7fe640 1 --2- 172.21.15.83:0/850350630 >> v2:172.21.15.19:3300/0 conn(0x7f738c0b3990 0x7f738c0af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.715+0000 7f738b7fe640 1 -- 172.21.15.83:0/850350630 >> v1:172.21.15.19:6789/0 conn(0x7f738c141a40 legacy=0x7f738c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:24.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.715+0000 7f738b7fe640 10 monclient: _finish_hunting 0 2023-10-06T13:52:24.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.715+0000 7f738b7fe640 1 monclient: found mon.noname-c 2023-10-06T13:52:24.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.715+0000 7f738b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.715+0000 7f738b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:52:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.715+0000 7f738b7fe640 1 -- 172.21.15.83:0/850350630 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c0aee30 con 0x7f738c0b3d60 2023-10-06T13:52:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.716+0000 7f738b7fe640 1 --2- 172.21.15.83:0/850350630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0b3d60 0x7f738c141500 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f738c06a5f0 tx=0x7f737c02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=412e7c2fa0fd7db0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:24.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f738affd640 1 -- 172.21.15.83:0/850350630 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c031870 con 0x7f738c0b3d60 2023-10-06T13:52:24.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f738affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:24.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f738affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:52:24.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f738affd640 10 monclient: dump: 2023-10-06T13:52:24.722 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:24.722 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:24.722 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.722 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.722 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:24.722 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:24.722 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:24.722 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:24.722 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:24.722 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f738affd640 10 monclient: _finish_auth 0 2023-10-06T13:52:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f738affd640 10 monclient: _check_auth_tickets 2023-10-06T13:52:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f738affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f738affd640 1 -- 172.21.15.83:0/850350630 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c0319d0 con 0x7f738c0b3d60 2023-10-06T13:52:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f738affd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f738affd640 1 -- 172.21.15.83:0/850350630 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c031d60 con 0x7f738c0b3d60 2023-10-06T13:52:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f738affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f738affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f738affd640 10 monclient: dump: 2023-10-06T13:52:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:24.724 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f739240b640 5 monclient: authenticate success, global_id 4150 2023-10-06T13:52:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f739240b640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f739240b640 4 set_mon_vals no callback set 2023-10-06T13:52:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f739240b640 10 monclient: shutdown 2023-10-06T13:52:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f739240b640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f739240b640 1 -- 172.21.15.83:0/850350630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0b3d60 msgr2=0x7f738c141500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.720+0000 7f739240b640 1 --2- 172.21.15.83:0/850350630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0b3d60 0x7f738c141500 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f738c06a5f0 tx=0x7f737c02b910 comp rx=0 tx=0).stop 2023-10-06T13:52:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.721+0000 7f739240b640 1 -- 172.21.15.83:0/850350630 shutdown_connections 2023-10-06T13:52:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.721+0000 7f739240b640 1 --2- 172.21.15.83:0/850350630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0b3d60 0x7f738c141500 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.721+0000 7f739240b640 1 --2- 172.21.15.83:0/850350630 >> v2:172.21.15.19:3300/0 conn(0x7f738c0b3990 0x7f738c0af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.721+0000 7f739240b640 1 -- 172.21.15.83:0/850350630 >> 172.21.15.83:0/850350630 conn(0x7f738c0b6e00 msgr2=0x7f738c0b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.721+0000 7f739240b640 1 -- 172.21.15.83:0/850350630 shutdown_connections 2023-10-06T13:52:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.721+0000 7f739240b640 1 -- 172.21.15.83:0/850350630 wait complete. 2023-10-06T13:52:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.721+0000 7f739240b640 10 monclient: build_initial_monmap 2023-10-06T13:52:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.721+0000 7f739240b640 10 monclient: monmap: 2023-10-06T13:52:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 0 2023-10-06T13:52:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 0.000000 2023-10-06T13:52:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:created 0.000000 2023-10-06T13:52:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:24.729 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.722+0000 7f739240b640 1 librados: starting msgr at 2023-10-06T13:52:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.722+0000 7f739240b640 1 librados: starting objecter 2023-10-06T13:52:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.722+0000 7f739240b640 1 Processor -- start 2023-10-06T13:52:24.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.722+0000 7f739240b640 1 -- start start 2023-10-06T13:52:24.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.722+0000 7f739240b640 1 librados: setting wanted keys 2023-10-06T13:52:24.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.722+0000 7f739240b640 1 librados: calling monclient init 2023-10-06T13:52:24.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.722+0000 7f739240b640 10 monclient: init 2023-10-06T13:52:24.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:24.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T13:52:24.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0b3990 0x7f738c0ac540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:24.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 10 monclient(hunting): picked mon.noname-b con 0x7f738c0b3990 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:24.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f738c0b3d60 0x7f738c120770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:24.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 10 monclient(hunting): picked mon.noname-a con 0x7f738c0b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:24.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f738c141a40 0x7f738c1d5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:24.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 10 monclient(hunting): picked mon.noname-c con 0x7f738c141a40 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c120d90 con 0x7f738c0b3d60 2023-10-06T13:52:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0ad080 con 0x7f738c0b3990 2023-10-06T13:52:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c1212b0 con 0x7f738c141a40 2023-10-06T13:52:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f739240b640 10 monclient(hunting): authenticate will time out at 1273.055405s 2023-10-06T13:52:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f738bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0b3990 0x7f738c0ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f738b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f738c0b3d60 0x7f738c120770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f73909a8640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f738c141a40 0x7f738c1d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f738bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0b3990 0x7f738c0ac540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52344/0 (socket says 172.21.15.83:52344) 2023-10-06T13:52:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f738bfff640 1 -- 172.21.15.83:0/2633389535 learned_addr learned my addr 172.21.15.83:0/2633389535 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-10-06T13:52:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f738bfff640 10 monclient(hunting): get_auth_request con 0x7f738c0b3990 auth_method 0 2023-10-06T13:52:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f738bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f738bfff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.723+0000 7f738bfff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.724+0000 7f738b7fe640 10 monclient(hunting): get_auth_request con 0x7f738c0b3d60 auth_method 0 2023-10-06T13:52:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.724+0000 7f738b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.724+0000 7f738b7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.724+0000 7f738b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.724+0000 7f73909a8640 10 monclient(hunting): get_auth_request con 0x7f738c141a40 auth_method 0 2023-10-06T13:52:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.724+0000 7f73909a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.724+0000 7f73909a8640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.724+0000 7f73909a8640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.724+0000 7f738b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.724+0000 7f738b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.724+0000 7f738b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.724+0000 7f73909a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.724+0000 7f73909a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.724+0000 7f73909a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.726+0000 7f738bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.726+0000 7f738bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.726+0000 7f738bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.727+0000 7f738b7fe640 10 monclient(hunting): handle_auth_done global_id 4158 payload 824 2023-10-06T13:52:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.727+0000 7f738b7fe640 1 -- 172.21.15.83:0/2633389535 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f738c141a40 msgr2=0x7f738c1d5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.727+0000 7f738b7fe640 1 --2- 172.21.15.83:0/2633389535 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f738c141a40 0x7f738c1d5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.727+0000 7f738b7fe640 1 -- 172.21.15.83:0/2633389535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0b3990 msgr2=0x7f738c0ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.727+0000 7f738b7fe640 1 --2- 172.21.15.83:0/2633389535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0b3990 0x7f738c0ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.727+0000 7f738b7fe640 10 monclient: _finish_hunting 0 2023-10-06T13:52:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.727+0000 7f738b7fe640 1 monclient: found mon.noname-a 2023-10-06T13:52:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.727+0000 7f738b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.727+0000 7f738b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:52:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.727+0000 7f738b7fe640 1 -- 172.21.15.83:0/2633389535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c0094d0 con 0x7f738c0b3d60 2023-10-06T13:52:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.729+0000 7f738b7fe640 1 --2- 172.21.15.83:0/2633389535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f738c0b3d60 0x7f738c120770 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f737c033000 tx=0x7f737c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.729+0000 7f7388ff9640 10 client.?.objecter ms_handle_connect 0x7f738c0b3d60 2023-10-06T13:52:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.729+0000 7f7388ff9640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.732+0000 7f7388ff9640 1 -- 172.21.15.83:0/2633389535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c03ac60 con 0x7f738c0b3d60 2023-10-06T13:52:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.732+0000 7f7388ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.732+0000 7f7388ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:52:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.732+0000 7f7388ff9640 10 monclient: dump: 2023-10-06T13:52:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:24.737 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.732+0000 7f7388ff9640 10 monclient: _finish_auth 0 2023-10-06T13:52:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.732+0000 7f7388ff9640 10 monclient: _check_auth_tickets 2023-10-06T13:52:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.732+0000 7f7388ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.732+0000 7f7388ff9640 1 -- 172.21.15.83:0/2633389535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c03ade0 con 0x7f738c0b3d60 2023-10-06T13:52:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.732+0000 7f739240b640 5 monclient: authenticate success, global_id 4158 2023-10-06T13:52:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.732+0000 7f739240b640 10 monclient: _renew_subs 2023-10-06T13:52:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.732+0000 7f739240b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:52:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f739240b640 1 -- 172.21.15.83:0/2633389535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738c120350 con 0x7f738c0b3d60 2023-10-06T13:52:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f7388ff9640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f7388ff9640 1 -- 172.21.15.83:0/2633389535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c044630 con 0x7f738c0b3d60 2023-10-06T13:52:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f7388ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f7388ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:52:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f7388ff9640 10 monclient: dump: 2023-10-06T13:52:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:epoch 1 2023-10-06T13:52:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:election_strategy: 1 2023-10-06T13:52:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:24.739 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-10-06T13:52:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f738a7fc640 4 set_mon_vals no callback set 2023-10-06T13:52:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f739240b640 10 client.4158.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f739240b640 10 monclient: _renew_subs 2023-10-06T13:52:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f739240b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:52:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f739240b640 1 -- 172.21.15.83:0/2633389535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c1d62a0 con 0x7f738c0b3d60 2023-10-06T13:52:24.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f739240b640 1 librados: init done 2023-10-06T13:52:24.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f739240b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:24.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f739240b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:24.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f739240b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:52:24.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.733+0000 7f739240b640 1 -- 172.21.15.83:0/2633389535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738c0678e0 con 0x7f738c0b3d60 2023-10-06T13:52:24.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.736+0000 7f7388ff9640 1 -- 172.21.15.83:0/2633389535 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f737c044a90 con 0x7f738c0b3d60 2023-10-06T13:52:24.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.736+0000 7f7388ff9640 1 -- 172.21.15.83:0/2633389535 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f737c02d040 con 0x7f738c0b3d60 2023-10-06T13:52:24.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.736+0000 7f7388ff9640 10 client.4158.objecter ms_dispatch 0x7f738c000b90 osd_map(9..9 src has 1..9) v4 2023-10-06T13:52:24.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.736+0000 7f7388ff9640 3 client.4158.objecter handle_osd_map got epochs [9,9] > 0 2023-10-06T13:52:24.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.736+0000 7f7388ff9640 3 client.4158.objecter handle_osd_map decoding full epoch 9 2023-10-06T13:52:24.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.736+0000 7f7388ff9640 20 client.4158.objecter dump_active .. 0 homeless 2023-10-06T13:52:24.767 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:24.766+0000 7f6f00a18040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T13:52:24.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.839+0000 7f7388ff9640 1 -- 172.21.15.83:0/2633389535 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f737c035020 con 0x7f738c0b3d60 2023-10-06T13:52:24.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.839+0000 7f7388ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:24.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.839+0000 7f7388ff9640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:24.868 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:24.867+0000 7f6f00a18040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T13:52:24.893 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:24.890+0000 7f4569a0e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T13:52:24.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.911+0000 7f739240b640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "9797eb84-1b2c-4280-ab13-bdb7b2007dd8", "id": 7}] 2023-10-06T13:52:24.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.911+0000 7f739240b640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "9797eb84-1b2c-4280-ab13-bdb7b2007dd8", "id": 7}] 2023-10-06T13:52:24.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.911+0000 7f739240b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:52:24.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:24.911+0000 7f739240b640 1 -- 172.21.15.83:0/2633389535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9797eb84-1b2c-4280-ab13-bdb7b2007dd8", "id": 7} v 0) v1 -- 0x7f738c0ae200 con 0x7f738c0b3d60 2023-10-06T13:52:24.949 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:24.947+0000 7f4569a0e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T13:52:25.095 INFO:tasks.ceph.mgr.y.smithi019.stderr:2023-10-06T13:52:25.093+0000 7f4569a0e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T13:52:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.100+0000 7f7388ff9640 1 -- 172.21.15.83:0/2633389535 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9797eb84-1b2c-4280-ab13-bdb7b2007dd8", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f737c005990 con 0x7f738c0b3d60 2023-10-06T13:52:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.100+0000 7f7388ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "9797eb84-1b2c-4280-ab13-bdb7b2007dd8", "id": 7}] 2023-10-06T13:52:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.100+0000 7f7388ff9640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:52:25.102 INFO:teuthology.orchestra.run.smithi083.stdout:7 2023-10-06T13:52:25.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 10 librados: watch_flush enter 2023-10-06T13:52:25.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 10 librados: watch_flush exit 2023-10-06T13:52:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 20 client.4158.objecter shutdown clearing up homeless session... 2023-10-06T13:52:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 10 client.4158.objecter successfully canceled tick 2023-10-06T13:52:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 10 monclient: shutdown 2023-10-06T13:52:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 1 -- 172.21.15.83:0/2633389535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f738c0b3d60 msgr2=0x7f738c120770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 1 --2- 172.21.15.83:0/2633389535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f738c0b3d60 0x7f738c120770 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f737c033000 tx=0x7f737c0046c0 comp rx=0 tx=0).stop 2023-10-06T13:52:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 1 -- 172.21.15.83:0/2633389535 shutdown_connections 2023-10-06T13:52:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 1 --2- 172.21.15.83:0/2633389535 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f738c141a40 0x7f738c1d5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 1 --2- 172.21.15.83:0/2633389535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f738c0b3d60 0x7f738c120770 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 1 --2- 172.21.15.83:0/2633389535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0b3990 0x7f738c0ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 1 -- 172.21.15.83:0/2633389535 >> 172.21.15.83:0/2633389535 conn(0x7f738c0b6e00 msgr2=0x7f738c0afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 1 -- 172.21.15.83:0/2633389535 shutdown_connections 2023-10-06T13:52:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 1 -- 172.21.15.83:0/2633389535 wait complete. 2023-10-06T13:52:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-10-06T13:52:25.101+0000 7f739240b640 1 librados: shutdown 2023-10-06T13:52:25.113 DEBUG:tasks.ceph_manager:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-06T13:52:25.113 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T13:52:25.113 DEBUG:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 2023-10-06T13:52:25.116 INFO:tasks.ceph.osd.0:Started 2023-10-06T13:52:25.116 DEBUG:tasks.ceph_manager:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-06T13:52:25.116 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T13:52:25.116 DEBUG:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 2023-10-06T13:52:25.118 INFO:tasks.ceph.osd.1:Started 2023-10-06T13:52:25.118 DEBUG:tasks.ceph_manager:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-06T13:52:25.119 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T13:52:25.119 DEBUG:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 2023-10-06T13:52:25.121 INFO:tasks.ceph.osd.2:Started 2023-10-06T13:52:25.121 DEBUG:tasks.ceph_manager:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-06T13:52:25.121 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T13:52:25.121 DEBUG:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 2023-10-06T13:52:25.124 INFO:tasks.ceph.osd.3:Started 2023-10-06T13:52:25.124 DEBUG:tasks.ceph_manager:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-06T13:52:25.124 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T13:52:25.124 DEBUG:teuthology.orchestra.run.smithi083:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 2023-10-06T13:52:25.126 INFO:tasks.ceph.osd.4:Started 2023-10-06T13:52:25.126 DEBUG:tasks.ceph_manager:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-06T13:52:25.126 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T13:52:25.127 DEBUG:teuthology.orchestra.run.smithi083:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 2023-10-06T13:52:25.128 INFO:tasks.ceph.osd.5:Started 2023-10-06T13:52:25.128 DEBUG:tasks.ceph_manager:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-06T13:52:25.129 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T13:52:25.129 DEBUG:teuthology.orchestra.run.smithi083:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 2023-10-06T13:52:25.131 INFO:tasks.ceph.osd.6:Started 2023-10-06T13:52:25.131 DEBUG:tasks.ceph_manager:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-06T13:52:25.131 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T13:52:25.131 DEBUG:teuthology.orchestra.run.smithi083:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 2023-10-06T13:52:25.133 INFO:tasks.ceph.osd.7:Started 2023-10-06T13:52:25.134 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T13:52:25.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.304+0000 7ff6c09dd640 10 monclient: get_monmap_and_config 2023-10-06T13:52:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.304+0000 7ff6c09dd640 10 monclient: build_initial_monmap 2023-10-06T13:52:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.304+0000 7ff6c09dd640 10 monclient: monmap: 2023-10-06T13:52:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:52:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:52:25.305584+0000 2023-10-06T13:52:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:52:25.305584+0000 2023-10-06T13:52:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:52:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:52:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:52:25.308 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:52:25.308 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:52:25.308 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:52:25.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.305+0000 7ff6c09dd640 1 Processor -- start 2023-10-06T13:52:25.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.305+0000 7ff6c09dd640 1 -- start start 2023-10-06T13:52:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.305+0000 7ff6c09dd640 10 monclient: init 2023-10-06T13:52:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.306+0000 7ff6c09dd640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.306+0000 7ff6c09dd640 10 monclient: _add_conns ranks=[0,1,2,4,3] 2023-10-06T13:52:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.306+0000 7ff6c09dd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff6bc0b3990 addr v1:172.21.15.19:6789/0 2023-10-06T13:52:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.306+0000 7ff6c09dd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff6bc0b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T13:52:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.306+0000 7ff6c09dd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff6bc0b07d0 0x7ff6bc0af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.306+0000 7ff6c09dd640 10 monclient(hunting): picked mon.noname-b con 0x7ff6bc0b07d0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.306+0000 7ff6c09dd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6bc0aebe0 con 0x7ff6bc0b3990 2023-10-06T13:52:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.306+0000 7ff6c09dd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6bc141070 con 0x7ff6bc0b3d60 2023-10-06T13:52:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.306+0000 7ff6c09dd640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.306+0000 7ff6c09dd640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc140dd0 con 0x7ff6bc0b07d0 2023-10-06T13:52:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.306+0000 7ff6c09dd640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:25.311 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.306+0000 7ff6c09dd640 10 monclient(hunting): authenticate will time out at 1273.493179s 2023-10-06T13:52:25.326 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.324+0000 7ff6ba59c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff6bc0b3990 0x7ff6bc141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35494/0 (socket says 172.21.15.19:35494) 2023-10-06T13:52:25.326 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.324+0000 7ff6ba59c640 1 -- 172.21.15.19:0/2211879006 learned_addr learned my addr 172.21.15.19:0/2211879006 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:52:25.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.338+0000 7ff6b959a640 1 -- 172.21.15.19:0/2211879006 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967096765 0 0) 0x7ff6bc0aebe0 con 0x7ff6bc0b3990 2023-10-06T13:52:25.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.338+0000 7ff6b959a640 10 monclient(hunting): _negotiate 2023-10-06T13:52:25.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.338+0000 7ff6b959a640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:25.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.338+0000 7ff6b959a640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:25.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.338+0000 7ff6b959a640 10 monclient(hunting): my global_id is 4161 2023-10-06T13:52:25.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.338+0000 7ff6b959a640 1 -- 172.21.15.19:0/2211879006 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a4003520 con 0x7ff6bc0b3990 2023-10-06T13:52:25.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 -- 172.21.15.19:0/2211879006 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2084723850 0 0) 0x7ff6a4003520 con 0x7ff6bc0b3990 2023-10-06T13:52:25.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 -- 172.21.15.19:0/2211879006 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff6bc0b07d0 msgr2=0x7ff6bc0af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:25.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 --2- 172.21.15.19:0/2211879006 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff6bc0b07d0 0x7ff6bc0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:25.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 -- 172.21.15.19:0/2211879006 >> v1:172.21.15.83:6789/0 conn(0x7ff6bc0b3d60 legacy=0x7ff6bc0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:25.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient: _finish_hunting 0 2023-10-06T13:52:25.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 monclient: found mon.noname-a-legacy 2023-10-06T13:52:25.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:25.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.19:6789/0 2023-10-06T13:52:25.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 -- 172.21.15.19:0/2211879006 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc141c10 con 0x7ff6bc0b3990 2023-10-06T13:52:25.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient: _finish_auth 0 2023-10-06T13:52:25.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient: _check_auth_tickets 2023-10-06T13:52:25.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:25.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 -- 172.21.15.19:0/2211879006 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251531364 0 0) 0x7ff6ac001670 con 0x7ff6bc0b3990 2023-10-06T13:52:25.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:25.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2023-10-06T13:52:25.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient: dump: 2023-10-06T13:52:25.350 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:52:25.350 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:25.350 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:25.350 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:25.351 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:25.351 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:25.351 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:25.351 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:25.351 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:25.351 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:52:25.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] but i'm connected to v1:172.21.15.19:6789/0, reconnecting 2023-10-06T13:52:25.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:25.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 -- 172.21.15.19:0/2211879006 >> v1:172.21.15.19:6789/0 conn(0x7ff6bc0b3990 legacy=0x7ff6bc141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:25.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:52:25.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 --2- 172.21.15.19:0/2211879006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a40039a0 0x7ff6a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:25.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient(hunting): picked mon.a con 0x7ff6a40039a0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:25.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 --2- 172.21.15.19:0/2211879006 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff6a40047c0 0x7ff6a4004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:25.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient(hunting): picked mon.c con 0x7ff6a40047c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:25.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 --2- 172.21.15.19:0/2211879006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6a4005290 0x7ff6a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:25.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient(hunting): picked mon.b con 0x7ff6a4005290 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:25.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:25.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 -- 172.21.15.19:0/2211879006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc140dd0 con 0x7ff6a40039a0 2023-10-06T13:52:25.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:25.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 -- 172.21.15.19:0/2211879006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a400c120 con 0x7ff6a4005290 2023-10-06T13:52:25.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:25.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 1 -- 172.21.15.19:0/2211879006 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a4014af0 con 0x7ff6a40047c0 2023-10-06T13:52:25.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:25.353 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.344+0000 7ff6b959a640 10 monclient(hunting): _finish_auth 0 2023-10-06T13:52:25.353 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.346+0000 7ff6ba59c640 1 --2- 172.21.15.19:0/2211879006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a40039a0 0x7ff6a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:25.354 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.349+0000 7ff6ba59c640 10 monclient(hunting): get_auth_request con 0x7ff6a40039a0 auth_method 0 2023-10-06T13:52:25.354 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.349+0000 7ff6ba59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:25.354 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.349+0000 7ff6ba59c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:25.354 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.349+0000 7ff6ba59c640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.349+0000 7ff6bad9d640 1 --2- 172.21.15.19:0/2211879006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6a4005290 0x7ff6a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.350+0000 7ff6ba59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.350+0000 7ff6ba59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.350+0000 7ff6ba59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T13:52:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.350+0000 7ff6bad9d640 10 monclient(hunting): get_auth_request con 0x7ff6a4005290 auth_method 0 2023-10-06T13:52:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.350+0000 7ff6bad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.350+0000 7ff6bad9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.350+0000 7ff6bad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:52:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.351+0000 7ff6bad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.351+0000 7ff6bad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.351+0000 7ff6bad9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T13:52:25.356 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6ba59c640 10 monclient(hunting): handle_auth_done global_id 4161 payload 293 2023-10-06T13:52:25.356 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6ba59c640 1 -- 172.21.15.19:0/2211879006 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff6a40047c0 msgr2=0x7ff6a4004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:25.356 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6ba59c640 1 --2- 172.21.15.19:0/2211879006 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff6a40047c0 0x7ff6a4004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:25.356 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6ba59c640 1 -- 172.21.15.19:0/2211879006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6a4005290 msgr2=0x7ff6a400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:25.357 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6ba59c640 1 --2- 172.21.15.19:0/2211879006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6a4005290 0x7ff6a400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:25.357 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6ba59c640 10 monclient: _finish_hunting 0 2023-10-06T13:52:25.357 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6ba59c640 1 monclient: found mon.a 2023-10-06T13:52:25.357 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6ba59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:25.357 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6ba59c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:52:25.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6ba59c640 1 -- 172.21.15.19:0/2211879006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc141c10 con 0x7ff6a40039a0 2023-10-06T13:52:25.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6ba59c640 10 monclient: _finish_auth 0 2023-10-06T13:52:25.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6ba59c640 10 monclient: _check_auth_tickets 2023-10-06T13:52:25.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6ba59c640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:25.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6c09dd640 5 monclient: authenticate success, global_id 4161 2023-10-06T13:52:25.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.353+0000 7ff6c09dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:52:25.359 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.356+0000 7ff6ba59c640 1 --2- 172.21.15.19:0/2211879006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a40039a0 0x7ff6a4003d90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00c370 tx=0x7ff6a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbd50a8e501308d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.362+0000 7ff6b959a640 1 -- 172.21.15.19:0/2211879006 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac02cc20 con 0x7ff6a40039a0 2023-10-06T13:52:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.362+0000 7ff6b959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.362+0000 7ff6b959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:52:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.362+0000 7ff6b959a640 10 monclient: dump: 2023-10-06T13:52:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:52:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:25.365 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:52:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.362+0000 7ff6b959a640 1 -- 172.21.15.19:0/2211879006 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac02cd80 con 0x7ff6a40039a0 2023-10-06T13:52:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.362+0000 7ff6b959a640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.362+0000 7ff6b959a640 1 -- 172.21.15.19:0/2211879006 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac005af0 con 0x7ff6a40039a0 2023-10-06T13:52:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.362+0000 7ff6b959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.362+0000 7ff6b959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:52:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.362+0000 7ff6b959a640 10 monclient: dump: 2023-10-06T13:52:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:52:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:25.366 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:52:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.362+0000 7ff6c09dd640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.362+0000 7ff6c09dd640 4 set_mon_vals no callback set 2023-10-06T13:52:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.363+0000 7ff6c09dd640 10 monclient: shutdown 2023-10-06T13:52:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.363+0000 7ff6c09dd640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.363+0000 7ff6c09dd640 1 -- 172.21.15.19:0/2211879006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a40039a0 msgr2=0x7ff6a4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.363+0000 7ff6c09dd640 1 --2- 172.21.15.19:0/2211879006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a40039a0 0x7ff6a4003d90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00c370 tx=0x7ff6a40057f0 comp rx=0 tx=0).stop 2023-10-06T13:52:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.363+0000 7ff6bad9d640 1 -- 172.21.15.19:0/2211879006 reap_dead start 2023-10-06T13:52:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.364+0000 7ff6c09dd640 1 -- 172.21.15.19:0/2211879006 shutdown_connections 2023-10-06T13:52:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.364+0000 7ff6c09dd640 1 -- 172.21.15.19:0/2211879006 >> 172.21.15.19:0/2211879006 conn(0x7ff6bc0b6e00 msgr2=0x7ff6bc0b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.364+0000 7ff6c09dd640 1 -- 172.21.15.19:0/2211879006 shutdown_connections 2023-10-06T13:52:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.364+0000 7ff6c09dd640 1 -- 172.21.15.19:0/2211879006 wait complete. 2023-10-06T13:52:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.365+0000 7ff6c09dd640 10 monclient: build_initial_monmap 2023-10-06T13:52:25.369 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.365+0000 7ff6c09dd640 10 monclient: monmap: 2023-10-06T13:52:25.369 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:52:25.369 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:25.369 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:52:25.369 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:52:25.369 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:25.369 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:25.369 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:25.369 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:25.369 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:25.369 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:52:25.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.366+0000 7ff6c09dd640 1 librados: starting msgr at 2023-10-06T13:52:25.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.366+0000 7ff6c09dd640 1 librados: starting objecter 2023-10-06T13:52:25.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.366+0000 7ff6c09dd640 1 Processor -- start 2023-10-06T13:52:25.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.366+0000 7ff6c09dd640 1 -- start start 2023-10-06T13:52:25.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.366+0000 7ff6c09dd640 1 librados: setting wanted keys 2023-10-06T13:52:25.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.366+0000 7ff6c09dd640 1 librados: calling monclient init 2023-10-06T13:52:25.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.366+0000 7ff6c09dd640 10 monclient: init 2023-10-06T13:52:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 10 monclient: _add_conns ranks=[2,0,1] 2023-10-06T13:52:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff6a40039a0 0x7ff6bc137350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 10 monclient(hunting): picked mon.noname-c con 0x7ff6a40039a0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6bc141e30 0x7ff6bc133fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 10 monclient(hunting): picked mon.noname-a con 0x7ff6bc141e30 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:25.374 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6bc134520 0x7ff6bc1d4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:25.374 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 10 monclient(hunting): picked mon.noname-b con 0x7ff6bc134520 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:25.374 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:25.374 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc0b6840 con 0x7ff6bc141e30 2023-10-06T13:52:25.374 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:25.374 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc142280 con 0x7ff6bc134520 2023-10-06T13:52:25.374 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:25.374 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc134990 con 0x7ff6a40039a0 2023-10-06T13:52:25.374 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:25.374 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.367+0000 7ff6c09dd640 10 monclient(hunting): authenticate will time out at 1273.554011s 2023-10-06T13:52:25.375 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.368+0000 7ff6b9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6bc141e30 0x7ff6bc133fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:25.375 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.368+0000 7ff6bad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6bc134520 0x7ff6bc1d4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:25.375 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.368+0000 7ff6b9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6bc141e30 0x7ff6bc133fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52182/0 (socket says 172.21.15.19:52182) 2023-10-06T13:52:25.375 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.368+0000 7ff6b9d9b640 1 -- 172.21.15.19:0/3177637768 learned_addr learned my addr 172.21.15.19:0/3177637768 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:52:25.375 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.368+0000 7ff6b9d9b640 10 monclient(hunting): get_auth_request con 0x7ff6bc141e30 auth_method 0 2023-10-06T13:52:25.375 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.368+0000 7ff6b9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:25.375 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.368+0000 7ff6b9d9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:25.375 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.368+0000 7ff6b9d9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:25.375 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.368+0000 7ff6bad9d640 10 monclient(hunting): get_auth_request con 0x7ff6bc134520 auth_method 0 2023-10-06T13:52:25.375 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.368+0000 7ff6bad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:25.375 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.368+0000 7ff6bad9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:25.376 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.368+0000 7ff6bad9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:25.376 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.369+0000 7ff6bad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:25.376 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.369+0000 7ff6bad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:25.376 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.369+0000 7ff6bad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:25.376 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.369+0000 7ff6b9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:25.376 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.369+0000 7ff6b9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:25.376 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.369+0000 7ff6b9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:25.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.371+0000 7ff6bad9d640 10 monclient(hunting): handle_auth_done global_id 4156 payload 824 2023-10-06T13:52:25.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.371+0000 7ff6bad9d640 1 -- 172.21.15.19:0/3177637768 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff6a40039a0 msgr2=0x7ff6bc137350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:25.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.371+0000 7ff6bad9d640 1 --2- 172.21.15.19:0/3177637768 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff6a40039a0 0x7ff6bc137350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:25.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.371+0000 7ff6bad9d640 1 -- 172.21.15.19:0/3177637768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6bc141e30 msgr2=0x7ff6bc133fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:25.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.371+0000 7ff6bad9d640 1 --2- 172.21.15.19:0/3177637768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6bc141e30 0x7ff6bc133fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:25.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.371+0000 7ff6bad9d640 10 monclient: _finish_hunting 0 2023-10-06T13:52:25.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.371+0000 7ff6bad9d640 1 monclient: found mon.noname-b 2023-10-06T13:52:25.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.371+0000 7ff6bad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:25.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.371+0000 7ff6bad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:52:25.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.371+0000 7ff6bad9d640 1 -- 172.21.15.19:0/3177637768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc0b63f0 con 0x7ff6bc134520 2023-10-06T13:52:25.378 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.374+0000 7ff6bad9d640 1 --2- 172.21.15.19:0/3177637768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6bc134520 0x7ff6bc1d4c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc066900 tx=0x7ff6b400ea60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:25.378 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.374+0000 7ff6aaffd640 10 client.?.objecter ms_handle_connect 0x7ff6bc134520 2023-10-06T13:52:25.378 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.374+0000 7ff6aaffd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:25.380 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6aaffd640 1 -- 172.21.15.19:0/3177637768 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4003b70 con 0x7ff6bc134520 2023-10-06T13:52:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6aaffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:52:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6aaffd640 10 monclient: dump: 2023-10-06T13:52:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:52:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:25.382 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:52:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6aaffd640 10 monclient: _finish_auth 0 2023-10-06T13:52:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6aaffd640 10 monclient: _check_auth_tickets 2023-10-06T13:52:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6aaffd640 1 -- 172.21.15.19:0/3177637768 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b4003cd0 con 0x7ff6bc134520 2023-10-06T13:52:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6aaffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6aaffd640 1 -- 172.21.15.19:0/3177637768 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4004260 con 0x7ff6bc134520 2023-10-06T13:52:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6aaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6aaffd640 10 monclient: dump: 2023-10-06T13:52:25.383 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:52:25.383 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:25.383 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:25.383 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:25.383 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:25.383 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:25.383 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:25.383 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:25.383 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:25.383 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:52:25.383 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6c09dd640 5 monclient: authenticate success, global_id 4156 2023-10-06T13:52:25.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6b8d99640 4 set_mon_vals no callback set 2023-10-06T13:52:25.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6c09dd640 10 monclient: _renew_subs 2023-10-06T13:52:25.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6c09dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:25.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6c09dd640 1 -- 172.21.15.19:0/3177637768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6bc142a60 con 0x7ff6bc134520 2023-10-06T13:52:25.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6c09dd640 10 client.4156.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:25.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6c09dd640 10 monclient: _renew_subs 2023-10-06T13:52:25.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6c09dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:25.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6c09dd640 1 -- 172.21.15.19:0/3177637768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6bc1d53a0 con 0x7ff6bc134520 2023-10-06T13:52:25.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.378+0000 7ff6c09dd640 1 librados: init done 2023-10-06T13:52:25.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.379+0000 7ff6c09dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:25.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.379+0000 7ff6c09dd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:25.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.379+0000 7ff6c09dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:25.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.379+0000 7ff6c09dd640 1 -- 172.21.15.19:0/3177637768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6bc0678e0 con 0x7ff6bc134520 2023-10-06T13:52:25.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.390+0000 7ff6aaffd640 1 -- 172.21.15.19:0/3177637768 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 2) v1 ==== 44409+0+0 (secure 0 0 0) 0x7ff6b40043c0 con 0x7ff6bc134520 2023-10-06T13:52:25.493 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.491+0000 7ff6aaffd640 1 -- 172.21.15.19:0/3177637768 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7ff6b40146f0 con 0x7ff6bc134520 2023-10-06T13:52:25.493 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.491+0000 7ff6aaffd640 10 client.4156.objecter ms_dispatch 0x7ff6bc000b90 osd_map(10..10 src has 1..10) v4 2023-10-06T13:52:25.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.491+0000 7ff6aaffd640 3 client.4156.objecter handle_osd_map got epochs [10,10] > 0 2023-10-06T13:52:25.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.491+0000 7ff6aaffd640 3 client.4156.objecter handle_osd_map decoding full epoch 10 2023-10-06T13:52:25.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.491+0000 7ff6aaffd640 20 client.4156.objecter dump_active .. 0 homeless 2023-10-06T13:52:25.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.491+0000 7ff6aaffd640 1 -- 172.21.15.19:0/3177637768 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7ff6b4014510 con 0x7ff6bc134520 2023-10-06T13:52:25.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.491+0000 7ff6aaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:25.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.491+0000 7ff6aaffd640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:25.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.615+0000 7ff6c09dd640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:25.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.615+0000 7ff6c09dd640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:25.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.615+0000 7ff6c09dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:25.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.615+0000 7ff6c09dd640 1 -- 172.21.15.19:0/3177637768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6bc0ae200 con 0x7ff6bc134520 2023-10-06T13:52:25.705 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.701+0000 7ff6aaffd640 1 -- 172.21.15.19:0/3177637768 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7ff6b4014330 con 0x7ff6bc134520 2023-10-06T13:52:25.705 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.701+0000 7ff6aaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:25.706 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.701+0000 7ff6aaffd640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:52:25.706 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:52:25.706 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"fsid":"c5a90f7c-81ec-43a3-ab47-af6ce5729ec5","created":"2023-10-06T13:52:19.516184+0000","modified":"2023-10-06T13:52:24.916896+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T13:52:24.916896+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b6cb75c5-86fb-48b9-895c-fb1d72bd4746","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ba828cb4-a4f6-4fea-a02e-24d9cf597ea7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"06bcfab5-d2f1-4447-ba44-143398707ef4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ad31d4df-e78c-4e32-bfd4-3fe6e70dd428","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"931b23f8-80c3-4595-8931-2568be479cb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"70dd2f95-93eb-448a-a65d-3a09bdea85a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9797eb84-1b2c-4280-ab13-bdb7b2007dd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T13:52:25.707 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.702+0000 7ff6c09dd640 10 librados: watch_flush enter 2023-10-06T13:52:25.707 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.702+0000 7ff6c09dd640 10 librados: watch_flush exit 2023-10-06T13:52:25.707 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.702+0000 7ff6c09dd640 20 client.4156.objecter shutdown clearing up homeless session... 2023-10-06T13:52:25.707 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.702+0000 7ff6c09dd640 10 client.4156.objecter successfully canceled tick 2023-10-06T13:52:25.707 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.702+0000 7ff6c09dd640 10 monclient: shutdown 2023-10-06T13:52:25.707 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.702+0000 7ff6c09dd640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:25.707 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.702+0000 7ff6c09dd640 1 -- 172.21.15.19:0/3177637768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6bc134520 msgr2=0x7ff6bc1d4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:25.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.702+0000 7ff6c09dd640 1 --2- 172.21.15.19:0/3177637768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6bc134520 0x7ff6bc1d4c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc066900 tx=0x7ff6b400ea60 comp rx=0 tx=0).stop 2023-10-06T13:52:25.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.703+0000 7ff6c09dd640 1 -- 172.21.15.19:0/3177637768 shutdown_connections 2023-10-06T13:52:25.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.703+0000 7ff6c09dd640 1 --2- 172.21.15.19:0/3177637768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6bc134520 0x7ff6bc1d4c80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:25.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.703+0000 7ff6c09dd640 1 --2- 172.21.15.19:0/3177637768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6bc141e30 0x7ff6bc133fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:25.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.703+0000 7ff6c09dd640 1 --2- 172.21.15.19:0/3177637768 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff6a40039a0 0x7ff6bc137350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:25.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.703+0000 7ff6c09dd640 1 -- 172.21.15.19:0/3177637768 >> 172.21.15.19:0/3177637768 conn(0x7ff6bc0b6e00 msgr2=0x7ff6bc13c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:25.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.704+0000 7ff6c09dd640 1 -- 172.21.15.19:0/3177637768 shutdown_connections 2023-10-06T13:52:25.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.704+0000 7ff6c09dd640 1 -- 172.21.15.19:0/3177637768 wait complete. 2023-10-06T13:52:25.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:52:25.704+0000 7ff6c09dd640 1 librados: shutdown 2023-10-06T13:52:25.726 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T13:52:25.726 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T13:52:26.532 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T13:52:26.684 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.682+0000 7f11b23ce640 10 monclient: get_monmap_and_config 2023-10-06T13:52:26.685 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.682+0000 7f11b23ce640 10 monclient: build_initial_monmap 2023-10-06T13:52:26.685 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.682+0000 7f11b23ce640 10 monclient: monmap: 2023-10-06T13:52:26.685 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-10-06T13:52:26.685 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:26.685 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:52:26.683634+0000 2023-10-06T13:52:26.685 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:52:26.683634+0000 2023-10-06T13:52:26.685 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:26.685 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:26.685 INFO:teuthology.misc.health.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:52:26.685 INFO:teuthology.misc.health.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:52:26.685 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:52:26.686 INFO:teuthology.misc.health.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:52:26.686 INFO:teuthology.misc.health.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:52:26.686 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:26.687 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.683+0000 7f11b23ce640 1 Processor -- start 2023-10-06T13:52:26.688 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.683+0000 7f11b23ce640 1 -- start start 2023-10-06T13:52:26.688 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.683+0000 7f11b23ce640 10 monclient: init 2023-10-06T13:52:26.688 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.684+0000 7f11b23ce640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:26.688 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.684+0000 7f11b23ce640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2023-10-06T13:52:26.688 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.684+0000 7f11b23ce640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f11ac0b3a00 0x7f11ac13dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:26.688 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.684+0000 7f11b23ce640 10 monclient(hunting): picked mon.noname-b con 0x7f11ac0b3a00 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:26.688 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.684+0000 7f11b23ce640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f11ac13e320 addr v1:172.21.15.83:6789/0 2023-10-06T13:52:26.688 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.684+0000 7f11b23ce640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11ac0b1e10 0x7f11ac14b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:26.688 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.684+0000 7f11b23ce640 10 monclient(hunting): picked mon.noname-a con 0x7f11ac0b1e10 addr v2:172.21.15.19:3300/0 2023-10-06T13:52:26.688 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.684+0000 7f11b23ce640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ac0b3dd0 con 0x7f11ac13e320 2023-10-06T13:52:26.689 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.684+0000 7f11b23ce640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:26.689 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.684+0000 7f11b23ce640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ac0a93d0 con 0x7f11ac0b1e10 2023-10-06T13:52:26.689 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.684+0000 7f11b23ce640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:26.689 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.684+0000 7f11b23ce640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac144ab0 con 0x7f11ac0b3a00 2023-10-06T13:52:26.689 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.684+0000 7f11b23ce640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:26.689 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.684+0000 7f11b23ce640 10 monclient(hunting): authenticate will time out at 1274.871351s 2023-10-06T13:52:26.704 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.702+0000 7f11b096b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11ac0b1e10 0x7f11ac14b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:26.705 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.702+0000 7f11b096b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11ac0b1e10 0x7f11ac14b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52184/0 (socket says 172.21.15.19:52184) 2023-10-06T13:52:26.705 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.702+0000 7f11b096b640 1 -- 172.21.15.19:0/1487793626 learned_addr learned my addr 172.21.15.19:0/1487793626 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:52:26.705 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.702+0000 7f11b096b640 10 monclient(hunting): get_auth_request con 0x7f11ac0b1e10 auth_method 0 2023-10-06T13:52:26.705 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.702+0000 7f11b096b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:26.705 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.702+0000 7f11b096b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:26.705 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.702+0000 7f11b096b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:26.706 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.703+0000 7f11b096b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:26.706 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.703+0000 7f11b096b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:26.706 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.703+0000 7f11b096b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:26.707 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.705+0000 7f11b096b640 10 monclient(hunting): handle_auth_done global_id 4176 payload 274 2023-10-06T13:52:26.707 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.705+0000 7f11b096b640 1 -- 172.21.15.19:0/1487793626 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f11ac0b3a00 msgr2=0x7f11ac13dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:26.707 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.705+0000 7f11b096b640 1 --2- 172.21.15.19:0/1487793626 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f11ac0b3a00 0x7f11ac13dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:26.707 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.705+0000 7f11b096b640 1 -- 172.21.15.19:0/1487793626 >> v1:172.21.15.83:6789/0 conn(0x7f11ac13e320 legacy=0x7f11ac0b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:26.708 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.705+0000 7f11b096b640 10 monclient: _finish_hunting 0 2023-10-06T13:52:26.708 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.705+0000 7f11b096b640 1 monclient: found mon.noname-a 2023-10-06T13:52:26.708 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.705+0000 7f11b096b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:26.708 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.705+0000 7f11b096b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:52:26.708 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.705+0000 7f11b096b640 1 -- 172.21.15.19:0/1487793626 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac0a95d0 con 0x7f11ac0b1e10 2023-10-06T13:52:26.785 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:26.783+0000 7f11b096b640 1 --2- 172.21.15.19:0/1487793626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11ac0b1e10 0x7f11ac14b4a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009e50 tx=0x7f11a402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c68679d63344d4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:26.943 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:26.942+0000 7f6ea1b7d640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T13:52:26.957 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T13:52:26.955+0000 7f6ea1b7d640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T13:52:27.092 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11aaffd640 1 -- 172.21.15.19:0/1487793626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a402ca70 con 0x7f11ac0b1e10 2023-10-06T13:52:27.092 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:27.092 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11aaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:52:27.092 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11aaffd640 10 monclient: dump: 2023-10-06T13:52:27.093 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:27.093 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:27.093 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:27.093 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:27.093 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:27.093 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:27.093 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:27.093 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:27.093 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:27.093 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:27.093 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11aaffd640 10 monclient: _finish_auth 0 2023-10-06T13:52:27.094 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11aaffd640 10 monclient: _check_auth_tickets 2023-10-06T13:52:27.094 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:27.094 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11aaffd640 1 -- 172.21.15.19:0/1487793626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a402cbd0 con 0x7f11ac0b1e10 2023-10-06T13:52:27.094 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11b23ce640 5 monclient: authenticate success, global_id 4176 2023-10-06T13:52:27.094 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11aaffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:27.094 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11b23ce640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:27.094 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11aaffd640 1 -- 172.21.15.19:0/1487793626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a40343f0 con 0x7f11ac0b1e10 2023-10-06T13:52:27.094 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11b23ce640 4 set_mon_vals no callback set 2023-10-06T13:52:27.094 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:27.095 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11aaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:52:27.095 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.090+0000 7f11aaffd640 10 monclient: dump: 2023-10-06T13:52:27.095 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:27.095 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:27.095 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:27.095 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:27.095 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:27.095 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:27.095 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:27.095 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:27.095 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:27.095 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:27.100 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.091+0000 7f11b23ce640 10 monclient: shutdown 2023-10-06T13:52:27.100 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.091+0000 7f11b23ce640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:27.100 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.091+0000 7f11b23ce640 1 -- 172.21.15.19:0/1487793626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11ac0b1e10 msgr2=0x7f11ac14b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:27.100 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.091+0000 7f11b23ce640 1 --2- 172.21.15.19:0/1487793626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11ac0b1e10 0x7f11ac14b4a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009e50 tx=0x7f11a402bed0 comp rx=0 tx=0).stop 2023-10-06T13:52:27.100 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.091+0000 7f11b23ce640 1 -- 172.21.15.19:0/1487793626 shutdown_connections 2023-10-06T13:52:27.100 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.091+0000 7f11b23ce640 1 --2- 172.21.15.19:0/1487793626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11ac0b1e10 0x7f11ac14b4a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:27.100 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.091+0000 7f11b23ce640 1 --2- 172.21.15.19:0/1487793626 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f11ac0b3a00 0x7f11ac13dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:27.101 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.091+0000 7f11b23ce640 1 -- 172.21.15.19:0/1487793626 >> 172.21.15.19:0/1487793626 conn(0x7f11ac0659b0 msgr2=0x7f11ac065db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:27.101 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.091+0000 7f11b23ce640 1 -- 172.21.15.19:0/1487793626 shutdown_connections 2023-10-06T13:52:27.101 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.091+0000 7f11b23ce640 1 -- 172.21.15.19:0/1487793626 wait complete. 2023-10-06T13:52:27.101 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.092+0000 7f11b23ce640 10 monclient: build_initial_monmap 2023-10-06T13:52:27.101 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.092+0000 7f11b23ce640 10 monclient: monmap: 2023-10-06T13:52:27.101 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-10-06T13:52:27.101 INFO:teuthology.misc.health.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:27.101 INFO:teuthology.misc.health.smithi019.stderr:last_changed 0.000000 2023-10-06T13:52:27.101 INFO:teuthology.misc.health.smithi019.stderr:created 0.000000 2023-10-06T13:52:27.101 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:27.101 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:27.102 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:27.102 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:27.102 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:27.102 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:27.102 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.093+0000 7f11b23ce640 1 librados: starting msgr at 2023-10-06T13:52:27.102 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.093+0000 7f11b23ce640 1 librados: starting objecter 2023-10-06T13:52:27.102 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.093+0000 7f11b23ce640 1 Processor -- start 2023-10-06T13:52:27.102 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.093+0000 7f11b23ce640 1 -- start start 2023-10-06T13:52:27.102 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.093+0000 7f11b23ce640 1 librados: setting wanted keys 2023-10-06T13:52:27.102 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.093+0000 7f11b23ce640 1 librados: calling monclient init 2023-10-06T13:52:27.102 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.093+0000 7f11b23ce640 10 monclient: init 2023-10-06T13:52:27.103 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.094+0000 7f11b23ce640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:27.103 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.094+0000 7f11b23ce640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:52:27.103 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.094+0000 7f11b23ce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11ac0b1e10 0x7f11ac0a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:27.103 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.094+0000 7f11b23ce640 10 monclient(hunting): picked mon.noname-a con 0x7f11ac0b1e10 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:27.103 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.094+0000 7f11b23ce640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f11ac0b3a00 0x7f11ac0b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:27.103 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.094+0000 7f11b23ce640 10 monclient(hunting): picked mon.noname-c con 0x7f11ac0b3a00 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:27.103 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.094+0000 7f11b23ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11ac13e320 0x7f11ac0b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:27.103 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.094+0000 7f11b23ce640 10 monclient(hunting): picked mon.noname-b con 0x7f11ac13e320 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:27.103 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.095+0000 7f11b23ce640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:27.103 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.095+0000 7f11b23ce640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac0ab560 con 0x7f11ac0b1e10 2023-10-06T13:52:27.104 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.095+0000 7f11b23ce640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:27.104 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.095+0000 7f11b23ce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac13e9b0 con 0x7f11ac13e320 2023-10-06T13:52:27.104 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.096+0000 7f11ab7fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f11ac0b3a00 0x7f11ac0b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:27.104 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.096+0000 7f11ab7fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f11ac0b3a00 0x7f11ac0b02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:57142/0 (socket says 172.21.15.19:57142) 2023-10-06T13:52:27.104 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.096+0000 7f11ab7fe640 1 -- 172.21.15.19:0/3387426843 learned_addr learned my addr 172.21.15.19:0/3387426843 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:52:27.104 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.096+0000 7f11b096b640 1 --2- 172.21.15.19:0/3387426843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11ac13e320 0x7f11ac0b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:27.104 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.096+0000 7f11b23ce640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:27.104 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.096+0000 7f11b23ce640 1 -- 172.21.15.19:0/3387426843 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac0b0d50 con 0x7f11ac0b3a00 2023-10-06T13:52:27.104 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.096+0000 7f11b23ce640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:27.104 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.096+0000 7f11b23ce640 10 monclient(hunting): authenticate will time out at 1275.283284s 2023-10-06T13:52:27.105 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.097+0000 7f11ab7fe640 10 monclient(hunting): get_auth_request con 0x7f11ac0b3a00 auth_method 0 2023-10-06T13:52:27.105 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.097+0000 7f11ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:27.105 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.097+0000 7f11ab7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:27.105 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.097+0000 7f11ab7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:27.105 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.097+0000 7f11b096b640 10 monclient(hunting): get_auth_request con 0x7f11ac13e320 auth_method 0 2023-10-06T13:52:27.105 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.097+0000 7f11b096b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:27.106 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.097+0000 7f11b096b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:27.106 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.097+0000 7f11b096b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:27.106 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.097+0000 7f11b096b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:27.106 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.097+0000 7f11b096b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:27.106 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.097+0000 7f11b096b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:27.107 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.097+0000 7f11ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:27.107 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.097+0000 7f11ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:27.107 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.097+0000 7f11ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:27.109 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.101+0000 7f11b096b640 10 monclient(hunting): handle_auth_done global_id 4174 payload 824 2023-10-06T13:52:27.109 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.101+0000 7f11b096b640 1 -- 172.21.15.19:0/3387426843 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f11ac0b3a00 msgr2=0x7f11ac0b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:27.109 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.101+0000 7f11b096b640 1 --2- 172.21.15.19:0/3387426843 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f11ac0b3a00 0x7f11ac0b02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:27.110 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.102+0000 7f11b096b640 1 -- 172.21.15.19:0/3387426843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11ac0b1e10 msgr2=0x7f11ac0a9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:27.110 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.102+0000 7f11b096b640 1 --2- 172.21.15.19:0/3387426843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11ac0b1e10 0x7f11ac0a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:27.110 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.102+0000 7f11b096b640 10 monclient: _finish_hunting 0 2023-10-06T13:52:27.110 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.102+0000 7f11b096b640 1 monclient: found mon.noname-b 2023-10-06T13:52:27.110 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.102+0000 7f11b096b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:27.110 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.102+0000 7f11b096b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:52:27.110 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.102+0000 7f11b096b640 1 -- 172.21.15.19:0/3387426843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a4009b30 con 0x7f11ac13e320 2023-10-06T13:52:27.110 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.105+0000 7f11b096b640 1 --2- 172.21.15.19:0/3387426843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11ac13e320 0x7f11ac0b0810 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f11a4032000 tx=0x7f11a4004a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:27.111 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.105+0000 7f11a8ff9640 10 client.?.objecter ms_handle_connect 0x7f11ac13e320 2023-10-06T13:52:27.111 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.105+0000 7f11a8ff9640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:27.113 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.108+0000 7f11a8ff9640 1 -- 172.21.15.19:0/3387426843 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4009510 con 0x7f11ac13e320 2023-10-06T13:52:27.113 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.108+0000 7f11a8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:27.113 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.108+0000 7f11a8ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:52:27.113 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.108+0000 7f11a8ff9640 10 monclient: dump: 2023-10-06T13:52:27.113 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:27.113 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:27.113 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:27.113 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:27.113 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:27.113 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:27.114 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:27.114 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:27.114 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:27.114 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:27.114 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.108+0000 7f11a8ff9640 10 monclient: _finish_auth 0 2023-10-06T13:52:27.114 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.108+0000 7f11a8ff9640 10 monclient: _check_auth_tickets 2023-10-06T13:52:27.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.108+0000 7f11a8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:27.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.108+0000 7f11b23ce640 5 monclient: authenticate success, global_id 4174 2023-10-06T13:52:27.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.108+0000 7f11b23ce640 10 monclient: _renew_subs 2023-10-06T13:52:27.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.109+0000 7f11b23ce640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:27.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.109+0000 7f11b23ce640 1 -- 172.21.15.19:0/3387426843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ac0ab760 con 0x7f11ac13e320 2023-10-06T13:52:27.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.109+0000 7f11b23ce640 10 client.4174.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:27.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.109+0000 7f11b23ce640 10 monclient: _renew_subs 2023-10-06T13:52:27.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.109+0000 7f11b23ce640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:27.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.109+0000 7f11b23ce640 1 -- 172.21.15.19:0/3387426843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ac140600 con 0x7f11ac13e320 2023-10-06T13:52:27.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.109+0000 7f11b23ce640 1 librados: init done 2023-10-06T13:52:27.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.110+0000 7f11b23ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:27.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.111+0000 7f11b23ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:27.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.111+0000 7f11b23ce640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:27.117 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.111+0000 7f11b23ce640 1 -- 172.21.15.19:0/3387426843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ac1d5be0 con 0x7f11ac13e320 2023-10-06T13:52:27.148 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.144+0000 7f11a8ff9640 1 -- 172.21.15.19:0/3387426843 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a403e450 con 0x7f11ac13e320 2023-10-06T13:52:27.148 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.144+0000 7f11a8ff9640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:27.148 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.144+0000 7f11a8ff9640 1 -- 172.21.15.19:0/3387426843 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a403e5c0 con 0x7f11ac13e320 2023-10-06T13:52:27.148 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.144+0000 7f11a8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:27.148 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.144+0000 7f11a8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:27.149 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.144+0000 7f11a8ff9640 10 monclient: dump: 2023-10-06T13:52:27.149 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:27.149 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:27.149 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:27.149 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:27.150 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:27.638 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:27.639 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:27.639 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:27.639 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:27.639 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:27.639 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.144+0000 7f11aa7fc640 4 set_mon_vals no callback set 2023-10-06T13:52:27.640 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.146+0000 7f11a8ff9640 1 -- 172.21.15.19:0/3387426843 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 3) v1 ==== 88753+0+0 (secure 0 0 0) 0x7f11a403e720 con 0x7f11ac13e320 2023-10-06T13:52:27.640 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.146+0000 7f11a8ff9640 1 --2- 172.21.15.19:0/3387426843 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f119806c010 0x7f119806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:27.642 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.147+0000 7f11abfff640 1 --2- 172.21.15.19:0/3387426843 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f119806c010 0x7f119806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:27.643 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.147+0000 7f11abfff640 10 monclient: get_auth_request con 0x7f119806c010 auth_method 0 2023-10-06T13:52:27.643 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.147+0000 7f11abfff640 1 --2- 172.21.15.19:0/3387426843 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f119806c010 0x7f119806e4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f119c009710 tx=0x7f119c006c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:27.643 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.147+0000 7f11a8ff9640 10 client.4174.objecter ms_handle_connect 0x7f119806c010 2023-10-06T13:52:27.643 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.149+0000 7f11a8ff9640 1 -- 172.21.15.19:0/3387426843 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f11a407b650 con 0x7f11ac13e320 2023-10-06T13:52:27.643 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.149+0000 7f11a8ff9640 10 client.4174.objecter ms_dispatch 0x7f11ac000b90 osd_map(10..10 src has 1..10) v4 2023-10-06T13:52:27.644 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.149+0000 7f11a8ff9640 3 client.4174.objecter handle_osd_map got epochs [10,10] > 0 2023-10-06T13:52:27.644 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.149+0000 7f11a8ff9640 3 client.4174.objecter handle_osd_map decoding full epoch 10 2023-10-06T13:52:27.644 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.149+0000 7f11a8ff9640 20 client.4174.objecter dump_active .. 0 homeless 2023-10-06T13:52:27.644 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.578+0000 7f11a8ff9640 1 -- 172.21.15.19:0/3387426843 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f11a4077290 con 0x7f11ac13e320 2023-10-06T13:52:27.644 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.578+0000 7f11a8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:27.645 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.578+0000 7f11a8ff9640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:27.947 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.945+0000 7f11b23ce640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:28.157 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.945+0000 7f11b23ce640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:28.157 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.945+0000 7f11b23ce640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:28.157 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:27.945+0000 7f11b23ce640 1 -- 172.21.15.19:0/3387426843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11ac1d5b90 con 0x7f11ac13e320 2023-10-06T13:52:28.158 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.131+0000 7f11a8ff9640 1 -- 172.21.15.19:0/3387426843 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 4) v1 ==== 88859+0+0 (secure 0 0 0) 0x7f11a4077660 con 0x7f11ac13e320 2023-10-06T13:52:28.195 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.193+0000 7f11a8ff9640 1 -- 172.21.15.19:0/3387426843 <== mon.1 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f11a40a7640 con 0x7f11ac13e320 2023-10-06T13:52:28.195 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.193+0000 7f11a8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:28.195 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.193+0000 7f11a8ff9640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:52:28.196 INFO:teuthology.misc.health.smithi019.stdout: 2023-10-06T13:52:28.196 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":10,"fsid":"c5a90f7c-81ec-43a3-ab47-af6ce5729ec5","created":"2023-10-06T13:52:19.516184+0000","modified":"2023-10-06T13:52:24.916896+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T13:52:24.916896+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b6cb75c5-86fb-48b9-895c-fb1d72bd4746","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ba828cb4-a4f6-4fea-a02e-24d9cf597ea7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"06bcfab5-d2f1-4447-ba44-143398707ef4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ad31d4df-e78c-4e32-bfd4-3fe6e70dd428","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"931b23f8-80c3-4595-8931-2568be479cb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"70dd2f95-93eb-448a-a65d-3a09bdea85a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9797eb84-1b2c-4280-ab13-bdb7b2007dd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T13:52:28.198 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 10 librados: watch_flush enter 2023-10-06T13:52:28.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 10 librados: watch_flush exit 2023-10-06T13:52:28.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 20 client.4174.objecter shutdown clearing up homeless session... 2023-10-06T13:52:28.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 10 client.4174.objecter successfully canceled tick 2023-10-06T13:52:28.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 1 -- 172.21.15.19:0/3387426843 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f119806c010 msgr2=0x7f119806e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:28.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 1 --2- 172.21.15.19:0/3387426843 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f119806c010 0x7f119806e4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f119c009710 tx=0x7f119c006c90 comp rx=0 tx=0).stop 2023-10-06T13:52:28.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 10 monclient: shutdown 2023-10-06T13:52:28.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:28.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 1 -- 172.21.15.19:0/3387426843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11ac13e320 msgr2=0x7f11ac0b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:28.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 1 --2- 172.21.15.19:0/3387426843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11ac13e320 0x7f11ac0b0810 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f11a4032000 tx=0x7f11a4004a90 comp rx=0 tx=0).stop 2023-10-06T13:52:28.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 1 -- 172.21.15.19:0/3387426843 shutdown_connections 2023-10-06T13:52:28.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 1 --2- 172.21.15.19:0/3387426843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11ac13e320 0x7f11ac0b0810 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:28.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 1 --2- 172.21.15.19:0/3387426843 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f119806c010 0x7f119806e4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:28.200 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 1 --2- 172.21.15.19:0/3387426843 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f11ac0b3a00 0x7f11ac0b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:28.200 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 1 --2- 172.21.15.19:0/3387426843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11ac0b1e10 0x7f11ac0a9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:28.200 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 1 -- 172.21.15.19:0/3387426843 >> 172.21.15.19:0/3387426843 conn(0x7f11ac0659b0 msgr2=0x7f11ac1435f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:28.200 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.196+0000 7f11b23ce640 1 -- 172.21.15.19:0/3387426843 shutdown_connections 2023-10-06T13:52:28.200 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.197+0000 7f11b23ce640 1 -- 172.21.15.19:0/3387426843 wait complete. 2023-10-06T13:52:28.200 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:28.197+0000 7f11b23ce640 1 librados: shutdown 2023-10-06T13:52:28.214 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T13:52:32.961 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:07.769 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:33.025 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:07.833 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:33.127 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:07.929 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:33.192 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:07.994 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:33.196 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:07.992 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:33.257 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:08.053 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:33.469 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:08.280 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:33.535 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:08.346 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:33.590 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:08.369 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:33.592 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T13:52:33.584+0000 5dc2340 -1 Falling back to public interface 2023-10-06T13:52:33.652 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:08.431 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:33.778 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T13:52:33.770+0000 5dc2340 -1 Falling back to public interface 2023-10-06T13:52:33.821 INFO:tasks.ceph.osd.5.smithi083.stderr:2023-10-06T13:52:33.811+0000 5dc2340 -1 Falling back to public interface 2023-10-06T13:52:33.899 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:08.691 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:33.968 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:08.760 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.070 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:08.878 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.094 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:52:34.085+0000 5dc2340 -1 Falling back to public interface 2023-10-06T13:52:34.111 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:08.902 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.148 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:08.946 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.173 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:08.964 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.207 INFO:tasks.ceph.osd.3.smithi019.stderr:2023-10-06T13:52:34.199+0000 5dc2340 -1 Falling back to public interface 2023-10-06T13:52:34.209 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:09.007 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.229 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:09.031 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.287 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:09.083 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.426 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:09.235 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.514 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T13:52:34.507+0000 5dc2340 -1 Falling back to public interface 2023-10-06T13:52:34.546 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:09.357 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.585 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:09.387 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.635 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:09.431 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.653 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:09.432 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.755 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T13:52:34.747+0000 5dc2340 -1 Falling back to public interface 2023-10-06T13:52:34.760 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T13:52:34.752+0000 5dc2340 -1 Falling back to public interface 2023-10-06T13:52:34.781 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:09.589 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.894 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:09.705 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.945 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:09.748 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:34.976 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:09.768 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.005 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:09.801 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.019 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T13:52:35.036 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:09.815 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.071 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:09.880 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.110 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.108+0000 7efdcc8b4640 10 monclient: get_monmap_and_config 2023-10-06T13:52:35.110 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.108+0000 7efdcc8b4640 10 monclient: build_initial_monmap 2023-10-06T13:52:35.111 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.108+0000 7efdcc8b4640 10 monclient: monmap: 2023-10-06T13:52:35.111 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-10-06T13:52:35.111 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:35.111 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:52:35.109917+0000 2023-10-06T13:52:35.111 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:52:35.109917+0000 2023-10-06T13:52:35.111 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:35.111 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:35.111 INFO:teuthology.misc.health.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:52:35.111 INFO:teuthology.misc.health.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:52:35.111 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:52:35.111 INFO:teuthology.misc.health.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:52:35.111 INFO:teuthology.misc.health.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:52:35.112 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:35.113 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.109+0000 7efdcc8b4640 1 Processor -- start 2023-10-06T13:52:35.113 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.109+0000 7efdcc8b4640 1 -- start start 2023-10-06T13:52:35.114 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.109+0000 7efdcc8b4640 10 monclient: init 2023-10-06T13:52:35.114 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdcc8b4640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:35.114 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdcc8b4640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2023-10-06T13:52:35.114 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdcc8b4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efdc80a8450 0x7efdc8144130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:35.114 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdcc8b4640 10 monclient(hunting): picked mon.noname-c con 0x7efdc80a8450 addr v2:172.21.15.83:3300/0 2023-10-06T13:52:35.114 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdcc8b4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efdc80b3050 addr v1:172.21.15.19:6789/0 2023-10-06T13:52:35.114 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdcc8b4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7efdc80a8820 0x7efdc8144670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:35.114 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdcc8b4640 10 monclient(hunting): picked mon.noname-b con 0x7efdc80a8820 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:35.114 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdcc8b4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc80a98b0 con 0x7efdc80b3050 2023-10-06T13:52:35.114 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdcc8b4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:35.114 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdcc8b4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc80a9b90 con 0x7efdc80a8450 2023-10-06T13:52:35.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdcc8b4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:35.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdcc8b4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc80b2850 con 0x7efdc80a8820 2023-10-06T13:52:35.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdcc8b4640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:35.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdcc8b4640 10 monclient(hunting): authenticate will time out at 1283.296655s 2023-10-06T13:52:35.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdc6d9d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7efdc80a8820 0x7efdc8144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:35.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.110+0000 7efdc5d9b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efdc80b3050 0x7efdc80b0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34858/0 (socket says 172.21.15.19:34858) 2023-10-06T13:52:35.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.111+0000 7efdc5d9b640 1 -- 172.21.15.19:0/2267467966 learned_addr learned my addr 172.21.15.19:0/2267467966 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:52:35.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.111+0000 7efdc659c640 1 --2- 172.21.15.19:0/2267467966 >> v2:172.21.15.83:3300/0 conn(0x7efdc80a8450 0x7efdc8144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:35.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.111+0000 7efdc6d9d640 10 monclient(hunting): get_auth_request con 0x7efdc80a8820 auth_method 0 2023-10-06T13:52:35.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.111+0000 7efdc6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:35.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.111+0000 7efdc6d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:35.115 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.111+0000 7efdc6d9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:35.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.111+0000 7efdc659c640 10 monclient(hunting): get_auth_request con 0x7efdc80a8450 auth_method 0 2023-10-06T13:52:35.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.111+0000 7efdc659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:35.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.111+0000 7efdc659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:35.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.111+0000 7efdc659c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:35.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.111+0000 7efdc659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:35.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.111+0000 7efdc659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:35.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.111+0000 7efdc659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:35.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.112+0000 7efdc6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:35.116 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.112+0000 7efdc6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:35.117 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.112+0000 7efdc6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:35.117 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.113+0000 7efdc659c640 10 monclient(hunting): handle_auth_done global_id 4207 payload 274 2023-10-06T13:52:35.117 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.113+0000 7efdc659c640 1 -- 172.21.15.19:0/2267467966 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7efdc80a8820 msgr2=0x7efdc8144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:35.118 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.113+0000 7efdc659c640 1 --2- 172.21.15.19:0/2267467966 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7efdc80a8820 0x7efdc8144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:35.118 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.113+0000 7efdc659c640 1 -- 172.21.15.19:0/2267467966 >> v1:172.21.15.19:6789/0 conn(0x7efdc80b3050 legacy=0x7efdc80b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:35.118 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.113+0000 7efdc659c640 10 monclient: _finish_hunting 0 2023-10-06T13:52:35.118 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.113+0000 7efdc659c640 1 monclient: found mon.noname-c 2023-10-06T13:52:35.118 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.113+0000 7efdc659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:35.118 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.113+0000 7efdc659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:52:35.118 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.113+0000 7efdc659c640 1 -- 172.21.15.19:0/2267467966 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc80b2b30 con 0x7efdc80a8450 2023-10-06T13:52:35.118 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.115+0000 7efdc659c640 1 --2- 172.21.15.19:0/2267467966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc80a8450 0x7efdc8144130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009790 tx=0x7efdb802b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d95bcba6cdcbfe1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:35.123 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdc559a640 1 -- 172.21.15.19:0/2267467966 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8031770 con 0x7efdc80a8450 2023-10-06T13:52:35.123 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdc559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:35.123 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdc559a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:52:35.123 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdc559a640 10 monclient: dump: 2023-10-06T13:52:35.123 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:35.123 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:35.123 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:35.123 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:35.123 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:35.123 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:35.123 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:35.123 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:35.124 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:35.124 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:35.124 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdc559a640 10 monclient: _finish_auth 0 2023-10-06T13:52:35.124 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdc559a640 10 monclient: _check_auth_tickets 2023-10-06T13:52:35.124 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdc559a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:35.124 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdc559a640 1 -- 172.21.15.19:0/2267467966 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb80318d0 con 0x7efdc80a8450 2023-10-06T13:52:35.124 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdc559a640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:35.124 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdc559a640 1 -- 172.21.15.19:0/2267467966 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8031c60 con 0x7efdc80a8450 2023-10-06T13:52:35.124 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdcc8b4640 5 monclient: authenticate success, global_id 4207 2023-10-06T13:52:35.124 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdcc8b4640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:35.124 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdcc8b4640 4 set_mon_vals no callback set 2023-10-06T13:52:35.124 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdc559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:35.125 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdc559a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:35.125 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdc559a640 10 monclient: dump: 2023-10-06T13:52:35.125 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:35.125 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:35.125 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:35.125 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:35.125 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:35.125 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:35.125 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:35.125 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:35.125 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:35.125 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:35.126 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdcc8b4640 10 monclient: shutdown 2023-10-06T13:52:35.126 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdcc8b4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:35.126 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdcc8b4640 1 -- 172.21.15.19:0/2267467966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc80a8450 msgr2=0x7efdc8144130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:35.126 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.119+0000 7efdcc8b4640 1 --2- 172.21.15.19:0/2267467966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc80a8450 0x7efdc8144130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009790 tx=0x7efdb802b810 comp rx=0 tx=0).stop 2023-10-06T13:52:35.126 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.120+0000 7efdcc8b4640 1 -- 172.21.15.19:0/2267467966 shutdown_connections 2023-10-06T13:52:35.126 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.120+0000 7efdcc8b4640 1 --2- 172.21.15.19:0/2267467966 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7efdc80a8820 0x7efdc8144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:35.126 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.120+0000 7efdcc8b4640 1 --2- 172.21.15.19:0/2267467966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc80a8450 0x7efdc8144130 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:35.126 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.120+0000 7efdcc8b4640 1 -- 172.21.15.19:0/2267467966 >> 172.21.15.19:0/2267467966 conn(0x7efdc8071490 msgr2=0x7efdc8071890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:35.126 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.120+0000 7efdcc8b4640 1 -- 172.21.15.19:0/2267467966 shutdown_connections 2023-10-06T13:52:35.127 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.120+0000 7efdcc8b4640 1 -- 172.21.15.19:0/2267467966 wait complete. 2023-10-06T13:52:35.127 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.120+0000 7efdcc8b4640 10 monclient: build_initial_monmap 2023-10-06T13:52:35.127 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.120+0000 7efdcc8b4640 10 monclient: monmap: 2023-10-06T13:52:35.127 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-10-06T13:52:35.127 INFO:teuthology.misc.health.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:35.127 INFO:teuthology.misc.health.smithi019.stderr:last_changed 0.000000 2023-10-06T13:52:35.127 INFO:teuthology.misc.health.smithi019.stderr:created 0.000000 2023-10-06T13:52:35.127 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:35.127 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:35.127 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:35.127 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:35.128 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:35.128 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:35.128 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 1 librados: starting msgr at 2023-10-06T13:52:35.128 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 1 librados: starting objecter 2023-10-06T13:52:35.128 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 1 Processor -- start 2023-10-06T13:52:35.128 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 1 -- start start 2023-10-06T13:52:35.128 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 1 librados: setting wanted keys 2023-10-06T13:52:35.128 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 1 librados: calling monclient init 2023-10-06T13:52:35.128 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 10 monclient: init 2023-10-06T13:52:35.131 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:35.131 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T13:52:35.131 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdc80a8450 0x7efdc8140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:35.131 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 10 monclient(hunting): picked mon.noname-a con 0x7efdc80a8450 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:35.131 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc80a8820 0x7efdc81d94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:35.131 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 10 monclient(hunting): picked mon.noname-b con 0x7efdc80a8820 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:35.131 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7efdc80b3050 0x7efdc81d9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:35.131 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 10 monclient(hunting): picked mon.noname-c con 0x7efdc80b3050 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:35.131 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:35.131 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc81406a0 con 0x7efdc80a8450 2023-10-06T13:52:35.132 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:35.132 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.121+0000 7efdcc8b4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc8141e90 con 0x7efdc80a8820 2023-10-06T13:52:35.132 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdcc8b4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:35.132 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdcc8b4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc80ac0c0 con 0x7efdc80b3050 2023-10-06T13:52:35.132 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdcc8b4640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:35.132 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdcc8b4640 10 monclient(hunting): authenticate will time out at 1283.308375s 2023-10-06T13:52:35.132 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdc659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdc80a8450 0x7efdc8140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:35.132 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdc6d9d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7efdc80b3050 0x7efdc81d9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:35.132 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdc659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdc80a8450 0x7efdc8140160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60392/0 (socket says 172.21.15.19:60392) 2023-10-06T13:52:35.132 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdc659c640 1 -- 172.21.15.19:0/1084771306 learned_addr learned my addr 172.21.15.19:0/1084771306 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:52:35.132 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdc659c640 10 monclient(hunting): get_auth_request con 0x7efdc80a8450 auth_method 0 2023-10-06T13:52:35.132 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdc659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:35.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdc659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:35.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdc659c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:35.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdc5d9b640 1 --2- 172.21.15.19:0/1084771306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc80a8820 0x7efdc81d94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:35.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdc6d9d640 10 monclient(hunting): get_auth_request con 0x7efdc80b3050 auth_method 0 2023-10-06T13:52:35.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdc6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:35.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdc6d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:35.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.122+0000 7efdc6d9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:35.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.123+0000 7efdc5d9b640 10 monclient(hunting): get_auth_request con 0x7efdc80a8820 auth_method 0 2023-10-06T13:52:35.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.123+0000 7efdc5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:35.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.123+0000 7efdc5d9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:35.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.123+0000 7efdc5d9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:35.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.123+0000 7efdc659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:35.134 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.123+0000 7efdc659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:35.134 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.123+0000 7efdc659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:35.134 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.123+0000 7efdc5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:35.134 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.123+0000 7efdc5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:35.134 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.123+0000 7efdc5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:35.134 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.123+0000 7efdc6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:35.134 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.123+0000 7efdc6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:35.134 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.123+0000 7efdc6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:35.134 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.126+0000 7efdc5d9b640 10 monclient(hunting): handle_auth_done global_id 4210 payload 824 2023-10-06T13:52:35.134 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.126+0000 7efdc5d9b640 1 -- 172.21.15.19:0/1084771306 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7efdc80b3050 msgr2=0x7efdc81d9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:35.134 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.126+0000 7efdc5d9b640 1 --2- 172.21.15.19:0/1084771306 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7efdc80b3050 0x7efdc81d9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:35.135 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.126+0000 7efdc5d9b640 1 -- 172.21.15.19:0/1084771306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdc80a8450 msgr2=0x7efdc8140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:35.135 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.126+0000 7efdc5d9b640 1 --2- 172.21.15.19:0/1084771306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdc80a8450 0x7efdc8140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:35.135 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.126+0000 7efdc5d9b640 10 monclient: _finish_hunting 0 2023-10-06T13:52:35.135 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.126+0000 7efdc5d9b640 1 monclient: found mon.noname-b 2023-10-06T13:52:35.135 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.126+0000 7efdc5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:35.135 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.126+0000 7efdc5d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:52:35.135 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.126+0000 7efdc5d9b640 1 -- 172.21.15.19:0/1084771306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb80093a0 con 0x7efdc80a8820 2023-10-06T13:52:35.135 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.128+0000 7efdc5d9b640 1 --2- 172.21.15.19:0/1084771306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc80a8820 0x7efdc81d94c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00c470 tx=0x7efdbc001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:35.135 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.128+0000 7efdb6ffd640 10 client.?.objecter ms_handle_connect 0x7efdc80a8820 2023-10-06T13:52:35.135 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.128+0000 7efdb6ffd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:35.136 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdb6ffd640 1 -- 172.21.15.19:0/1084771306 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc005ce0 con 0x7efdc80a8820 2023-10-06T13:52:35.136 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdb6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:35.136 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdb6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:52:35.137 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdb6ffd640 10 monclient: dump: 2023-10-06T13:52:35.137 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:35.137 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:35.137 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:35.137 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:35.137 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:35.137 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:35.137 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:35.138 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:35.138 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:35.138 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:35.138 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdb6ffd640 10 monclient: _finish_auth 0 2023-10-06T13:52:35.138 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdb6ffd640 10 monclient: _check_auth_tickets 2023-10-06T13:52:35.138 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdb6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:35.138 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdb6ffd640 1 -- 172.21.15.19:0/1084771306 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdbc005e40 con 0x7efdc80a8820 2023-10-06T13:52:35.138 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdb6ffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:35.138 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdb6ffd640 1 -- 172.21.15.19:0/1084771306 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc0072d0 con 0x7efdc80a8820 2023-10-06T13:52:35.138 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdcc8b4640 5 monclient: authenticate success, global_id 4210 2023-10-06T13:52:35.138 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdcc8b4640 10 monclient: _renew_subs 2023-10-06T13:52:35.138 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdcc8b4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:35.138 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdcc8b4640 1 -- 172.21.15.19:0/1084771306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc8140ed0 con 0x7efdc80a8820 2023-10-06T13:52:35.139 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdb6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:35.139 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdc4d99640 4 set_mon_vals no callback set 2023-10-06T13:52:35.139 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdb6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:35.139 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdb6ffd640 10 monclient: dump: 2023-10-06T13:52:35.139 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:35.139 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:35.139 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:35.139 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:35.139 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:35.139 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:35.139 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:35.139 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:35.140 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:35.140 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:35.140 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdcc8b4640 10 client.4210.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:35.140 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdcc8b4640 10 monclient: _renew_subs 2023-10-06T13:52:35.140 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdcc8b4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:35.140 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdcc8b4640 1 -- 172.21.15.19:0/1084771306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc81d9f40 con 0x7efdc80a8820 2023-10-06T13:52:35.140 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.131+0000 7efdcc8b4640 1 librados: init done 2023-10-06T13:52:35.140 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.132+0000 7efdcc8b4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:35.140 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.132+0000 7efdcc8b4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:35.140 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.132+0000 7efdcc8b4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:35.140 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.132+0000 7efdcc8b4640 1 -- 172.21.15.19:0/1084771306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc804e000 con 0x7efdc80a8820 2023-10-06T13:52:35.151 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.149+0000 7efdb6ffd640 1 -- 172.21.15.19:0/1084771306 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 88859+0+0 (secure 0 0 0) 0x7efdbc007430 con 0x7efdc80a8820 2023-10-06T13:52:35.151 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.149+0000 7efdb6ffd640 1 --2- 172.21.15.19:0/1084771306 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7efda806c180 0x7efda806e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:35.152 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.150+0000 7efdc659c640 1 --2- 172.21.15.19:0/1084771306 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7efda806c180 0x7efda806e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:35.152 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.150+0000 7efdc659c640 10 monclient: get_auth_request con 0x7efda806c180 auth_method 0 2023-10-06T13:52:35.152 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.150+0000 7efdb6ffd640 10 client.4210.objecter ms_handle_connect 0x7efda806c180 2023-10-06T13:52:35.152 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.150+0000 7efdc659c640 1 --2- 172.21.15.19:0/1084771306 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7efda806c180 0x7efda806e630 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efdb802b850 tx=0x7efdb803d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:35.152 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.151+0000 7efdb6ffd640 1 -- 172.21.15.19:0/1084771306 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7efdbc057970 con 0x7efdc80a8820 2023-10-06T13:52:35.153 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.151+0000 7efdb6ffd640 10 client.4210.objecter ms_dispatch 0x7efdc8000b90 osd_map(10..10 src has 1..10) v4 2023-10-06T13:52:35.153 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.151+0000 7efdb6ffd640 3 client.4210.objecter handle_osd_map got epochs [10,10] > 0 2023-10-06T13:52:35.153 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.151+0000 7efdb6ffd640 3 client.4210.objecter handle_osd_map decoding full epoch 10 2023-10-06T13:52:35.153 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.151+0000 7efdb6ffd640 20 client.4210.objecter dump_active .. 0 homeless 2023-10-06T13:52:35.233 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:10.036 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.240 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:10.038 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.266 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:10.077 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.293 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:10.089 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.297 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:10.088 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.335 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:10.127 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.358 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:10.166 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.403 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:10.182 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.408 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.406+0000 7efdb6ffd640 1 -- 172.21.15.19:0/1084771306 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7efdbc053cb0 con 0x7efdc80a8820 2023-10-06T13:52:35.408 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.406+0000 7efdb6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:35.408 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.406+0000 7efdb6ffd640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:35.521 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:10.324 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.559 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:10.370 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.589 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.587+0000 7efdcc8b4640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:35.589 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.587+0000 7efdcc8b4640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:35.589 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.587+0000 7efdcc8b4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:35.589 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.587+0000 7efdcc8b4640 1 -- 172.21.15.19:0/1084771306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdc813aa10 con 0x7efdc80a8820 2023-10-06T13:52:35.591 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:10.389 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.596 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:10.392 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.604 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.602+0000 7efdb6ffd640 1 -- 172.21.15.19:0/1084771306 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7efdbc021090 con 0x7efdc80a8820 2023-10-06T13:52:35.604 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.602+0000 7efdb6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:35.604 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.602+0000 7efdb6ffd640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:52:35.604 INFO:teuthology.misc.health.smithi019.stdout: 2023-10-06T13:52:35.604 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":10,"fsid":"c5a90f7c-81ec-43a3-ab47-af6ce5729ec5","created":"2023-10-06T13:52:19.516184+0000","modified":"2023-10-06T13:52:24.916896+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T13:52:24.916896+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b6cb75c5-86fb-48b9-895c-fb1d72bd4746","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ba828cb4-a4f6-4fea-a02e-24d9cf597ea7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"06bcfab5-d2f1-4447-ba44-143398707ef4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ad31d4df-e78c-4e32-bfd4-3fe6e70dd428","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"931b23f8-80c3-4595-8931-2568be479cb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"70dd2f95-93eb-448a-a65d-3a09bdea85a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9797eb84-1b2c-4280-ab13-bdb7b2007dd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T13:52:35.607 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 10 librados: watch_flush enter 2023-10-06T13:52:35.607 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 10 librados: watch_flush exit 2023-10-06T13:52:35.607 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 20 client.4210.objecter shutdown clearing up homeless session... 2023-10-06T13:52:35.607 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 10 client.4210.objecter successfully canceled tick 2023-10-06T13:52:35.607 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 1 -- 172.21.15.19:0/1084771306 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7efda806c180 msgr2=0x7efda806e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:35.607 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 1 --2- 172.21.15.19:0/1084771306 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7efda806c180 0x7efda806e630 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efdb802b850 tx=0x7efdb803d040 comp rx=0 tx=0).stop 2023-10-06T13:52:35.608 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 10 monclient: shutdown 2023-10-06T13:52:35.608 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:35.608 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 1 -- 172.21.15.19:0/1084771306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc80a8820 msgr2=0x7efdc81d94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:35.608 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 1 --2- 172.21.15.19:0/1084771306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc80a8820 0x7efdc81d94c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00c470 tx=0x7efdbc001e70 comp rx=0 tx=0).stop 2023-10-06T13:52:35.608 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 1 -- 172.21.15.19:0/1084771306 shutdown_connections 2023-10-06T13:52:35.608 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 1 --2- 172.21.15.19:0/1084771306 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7efda806c180 0x7efda806e630 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:35.608 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 1 --2- 172.21.15.19:0/1084771306 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7efdc80b3050 0x7efdc81d9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:35.608 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 1 --2- 172.21.15.19:0/1084771306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc80a8820 0x7efdc81d94c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:35.609 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 1 --2- 172.21.15.19:0/1084771306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdc80a8450 0x7efdc8140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:35.609 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.604+0000 7efdcc8b4640 1 -- 172.21.15.19:0/1084771306 >> 172.21.15.19:0/1084771306 conn(0x7efdc8071490 msgr2=0x7efdc8066100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:35.609 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.605+0000 7efdcc8b4640 1 -- 172.21.15.19:0/1084771306 shutdown_connections 2023-10-06T13:52:35.609 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.605+0000 7efdcc8b4640 1 -- 172.21.15.19:0/1084771306 wait complete. 2023-10-06T13:52:35.609 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:35.605+0000 7efdcc8b4640 1 librados: shutdown 2023-10-06T13:52:35.616 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T13:52:35.649 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:10.440 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.650 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:10.459 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.705 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:10.484 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.732 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:10.524 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.813 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:10.615 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.849 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:10.660 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.891 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:10.687 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.942 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:10.750 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:35.953 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:10.751 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.008 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:10.787 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.036 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:10.828 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.050 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:10.840 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.100 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:10.902 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.138 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:10.949 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.179 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:10.974 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.229 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:11.038 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.242 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:11.040 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.281 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:11.060 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.305 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:11.097 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.339 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:11.130 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.367 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:11.169 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.402 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:11.213 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.443 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:11.238 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.494 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:11.302 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.508 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:11.305 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.546 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:11.325 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.574 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:11.367 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.612 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:11.403 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.633 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:11.436 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.667 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:11.478 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.708 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:11.503 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.772 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:11.570 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.815 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:11.594 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.840 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:11.633 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.878 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:11.668 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.942 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:11.753 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:36.959 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:11.767 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.013 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:11.822 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.043 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:11.841 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.085 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:11.864 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.116 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:11.908 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.122 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:11.925 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.148 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:11.938 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.177 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:11.980 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.210 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:12.006 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.265 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:12.061 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.340 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:12.138 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.419 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:12.211 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.447 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:12.237 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.454 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:12.265 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.509 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:12.320 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.615 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:12.394 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.634 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:12.432 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.671 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:12.450 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.737 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:12.527 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:37.956 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:12.748 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:38.058 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:12.850 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:38.216 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:13.014 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:38.606 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:13.052 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:38.606 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:13.071 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:38.607 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:13.129 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:42.199 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:17.008 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:42.422 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T13:52:42.462 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:17.258 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:42.557 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.555+0000 7f794daa4640 10 monclient: get_monmap_and_config 2023-10-06T13:52:42.557 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.555+0000 7f794daa4640 10 monclient: build_initial_monmap 2023-10-06T13:52:42.558 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.555+0000 7f794daa4640 10 monclient: monmap: 2023-10-06T13:52:42.558 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-10-06T13:52:42.558 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:42.558 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:52:42.556760+0000 2023-10-06T13:52:42.558 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:52:42.556760+0000 2023-10-06T13:52:42.558 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:42.558 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:42.558 INFO:teuthology.misc.health.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:52:42.558 INFO:teuthology.misc.health.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:52:42.558 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:52:42.559 INFO:teuthology.misc.health.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:52:42.559 INFO:teuthology.misc.health.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:52:42.559 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:42.560 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.556+0000 7f794daa4640 1 Processor -- start 2023-10-06T13:52:42.560 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.556+0000 7f794daa4640 1 -- start start 2023-10-06T13:52:42.561 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.556+0000 7f794daa4640 10 monclient: init 2023-10-06T13:52:42.561 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.557+0000 7f794daa4640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:42.561 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.557+0000 7f794daa4640 10 monclient: _add_conns ranks=[3,4,1,0,2] 2023-10-06T13:52:42.561 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.557+0000 7f794daa4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79480b3a00 0x7f794813dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:42.561 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.557+0000 7f794daa4640 10 monclient(hunting): picked mon.noname-a con 0x7f79480b3a00 addr v2:172.21.15.19:3300/0 2023-10-06T13:52:42.561 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.557+0000 7f794daa4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f794813e320 0x7f79480b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:42.561 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.557+0000 7f794daa4640 10 monclient(hunting): picked mon.noname-c con 0x7f794813e320 addr v2:172.21.15.83:3300/0 2023-10-06T13:52:42.561 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.557+0000 7f794daa4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f79480b1c40 addr v1:172.21.15.83:6789/0 2023-10-06T13:52:42.561 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.557+0000 7f794daa4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79480b3dd0 con 0x7f79480b1c40 2023-10-06T13:52:42.561 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.557+0000 7f794daa4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:42.561 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.557+0000 7f794daa4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79480a93d0 con 0x7f79480b3a00 2023-10-06T13:52:42.562 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.557+0000 7f794daa4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:42.562 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.557+0000 7f794daa4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7948144ab0 con 0x7f794813e320 2023-10-06T13:52:42.562 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.557+0000 7f794daa4640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:42.562 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.557+0000 7f794daa4640 10 monclient(hunting): authenticate will time out at 1290.744278s 2023-10-06T13:52:42.562 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.558+0000 7f79477fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79480b3a00 0x7f794813dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:42.562 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.558+0000 7f7946ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f794813e320 0x7f79480b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:42.562 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.558+0000 7f79477fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79480b3a00 0x7f794813dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60400/0 (socket says 172.21.15.19:60400) 2023-10-06T13:52:42.562 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.558+0000 7f79477fe640 1 -- 172.21.15.19:0/2478107869 learned_addr learned my addr 172.21.15.19:0/2478107869 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:52:42.562 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.558+0000 7f79477fe640 10 monclient(hunting): get_auth_request con 0x7f79480b3a00 auth_method 0 2023-10-06T13:52:42.562 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.558+0000 7f79477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:42.562 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.558+0000 7f79477fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:42.562 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.558+0000 7f79477fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:42.563 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.559+0000 7f7946ffd640 10 monclient(hunting): get_auth_request con 0x7f794813e320 auth_method 0 2023-10-06T13:52:42.563 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.559+0000 7f7946ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:42.563 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.559+0000 7f7946ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:42.563 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.559+0000 7f7946ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:42.564 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.559+0000 7f79477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:42.565 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.559+0000 7f79477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:42.565 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.559+0000 7f79477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:42.565 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.560+0000 7f7946ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:42.565 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.560+0000 7f7946ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:42.565 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.560+0000 7f7946ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:42.565 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.561+0000 7f79477fe640 10 monclient(hunting): handle_auth_done global_id 4203 payload 274 2023-10-06T13:52:42.565 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.561+0000 7f79477fe640 1 -- 172.21.15.19:0/2478107869 >> v2:172.21.15.83:3300/0 conn(0x7f794813e320 msgr2=0x7f79480b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:42.565 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.561+0000 7f79477fe640 1 --2- 172.21.15.19:0/2478107869 >> v2:172.21.15.83:3300/0 conn(0x7f794813e320 0x7f79480b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:42.565 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.561+0000 7f79477fe640 1 -- 172.21.15.19:0/2478107869 >> v1:172.21.15.83:6789/0 conn(0x7f79480b1c40 legacy=0x7f7948140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:42.565 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.562+0000 7f79477fe640 10 monclient: _finish_hunting 0 2023-10-06T13:52:42.565 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.562+0000 7f79477fe640 1 monclient: found mon.noname-a 2023-10-06T13:52:42.565 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.562+0000 7f79477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:42.566 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.562+0000 7f79477fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:52:42.566 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.562+0000 7f79477fe640 1 -- 172.21.15.19:0/2478107869 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79480a95d0 con 0x7f79480b3a00 2023-10-06T13:52:42.566 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.564+0000 7f79477fe640 1 --2- 172.21.15.19:0/2478107869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79480b3a00 0x7f794813dde0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f79480a8be0 tx=0x7f793802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8349a0b2a0c3a8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:42.574 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.569+0000 7f79467fc640 1 -- 172.21.15.19:0/2478107869 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938031870 con 0x7f79480b3a00 2023-10-06T13:52:42.574 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.569+0000 7f79467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:42.574 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.569+0000 7f79467fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:52:42.574 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.569+0000 7f79467fc640 10 monclient: dump: 2023-10-06T13:52:42.574 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:42.574 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:42.574 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:42.574 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:42.574 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:42.575 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:42.575 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:42.575 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:42.575 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:42.575 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:42.575 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.569+0000 7f79467fc640 10 monclient: _finish_auth 0 2023-10-06T13:52:42.575 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.569+0000 7f79467fc640 10 monclient: _check_auth_tickets 2023-10-06T13:52:42.575 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.569+0000 7f79467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:42.575 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.570+0000 7f79467fc640 1 -- 172.21.15.19:0/2478107869 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79380319d0 con 0x7f79480b3a00 2023-10-06T13:52:42.575 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.570+0000 7f794daa4640 5 monclient: authenticate success, global_id 4203 2023-10-06T13:52:42.575 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.570+0000 7f794daa4640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:52:42.575 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.570+0000 7f79467fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:42.576 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.570+0000 7f79467fc640 1 -- 172.21.15.19:0/2478107869 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938031d60 con 0x7f79480b3a00 2023-10-06T13:52:42.576 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.570+0000 7f794daa4640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:42.576 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.570+0000 7f794daa4640 4 set_mon_vals no callback set 2023-10-06T13:52:42.576 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.570+0000 7f79467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:42.576 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.570+0000 7f79467fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:52:42.576 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.570+0000 7f79467fc640 10 monclient: dump: 2023-10-06T13:52:42.576 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:42.576 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:42.576 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:42.576 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:42.576 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:42.577 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:42.577 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:42.577 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:42.577 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:42.577 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:42.577 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.570+0000 7f794daa4640 10 monclient: shutdown 2023-10-06T13:52:42.577 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.570+0000 7f794daa4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:42.577 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.570+0000 7f794daa4640 1 -- 172.21.15.19:0/2478107869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79480b3a00 msgr2=0x7f794813dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:42.577 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.570+0000 7f794daa4640 1 --2- 172.21.15.19:0/2478107869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79480b3a00 0x7f794813dde0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f79480a8be0 tx=0x7f793802b910 comp rx=0 tx=0).stop 2023-10-06T13:52:42.577 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.571+0000 7f794daa4640 1 -- 172.21.15.19:0/2478107869 shutdown_connections 2023-10-06T13:52:42.577 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.571+0000 7f794daa4640 1 --2- 172.21.15.19:0/2478107869 >> v2:172.21.15.83:3300/0 conn(0x7f794813e320 0x7f79480b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:42.577 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.571+0000 7f794daa4640 1 --2- 172.21.15.19:0/2478107869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79480b3a00 0x7f794813dde0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:42.578 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.571+0000 7f794daa4640 1 -- 172.21.15.19:0/2478107869 >> 172.21.15.19:0/2478107869 conn(0x7f79480659b0 msgr2=0x7f7948065db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:42.578 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.571+0000 7f794daa4640 1 -- 172.21.15.19:0/2478107869 shutdown_connections 2023-10-06T13:52:42.578 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.571+0000 7f794daa4640 1 -- 172.21.15.19:0/2478107869 wait complete. 2023-10-06T13:52:42.578 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.572+0000 7f794daa4640 10 monclient: build_initial_monmap 2023-10-06T13:52:42.578 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.572+0000 7f794daa4640 10 monclient: monmap: 2023-10-06T13:52:42.578 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-10-06T13:52:42.578 INFO:teuthology.misc.health.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:42.578 INFO:teuthology.misc.health.smithi019.stderr:last_changed 0.000000 2023-10-06T13:52:42.578 INFO:teuthology.misc.health.smithi019.stderr:created 0.000000 2023-10-06T13:52:42.578 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:42.578 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:42.579 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:42.579 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:42.579 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:42.579 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:42.582 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.572+0000 7f794daa4640 1 librados: starting msgr at 2023-10-06T13:52:42.582 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.572+0000 7f794daa4640 1 librados: starting objecter 2023-10-06T13:52:42.582 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.572+0000 7f794daa4640 1 Processor -- start 2023-10-06T13:52:42.582 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.573+0000 7f794daa4640 1 -- start start 2023-10-06T13:52:42.582 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.573+0000 7f794daa4640 1 librados: setting wanted keys 2023-10-06T13:52:42.582 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.573+0000 7f794daa4640 1 librados: calling monclient init 2023-10-06T13:52:42.582 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.573+0000 7f794daa4640 10 monclient: init 2023-10-06T13:52:42.583 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:42.583 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T13:52:42.583 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79480b1c40 0x7f79481403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:42.583 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 10 monclient(hunting): picked mon.noname-a con 0x7f79480b1c40 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:42.583 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79480b3a00 0x7f79481d90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:42.583 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 10 monclient(hunting): picked mon.noname-b con 0x7f79480b3a00 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:42.583 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f794813e320 0x7f79481382d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:42.583 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 10 monclient(hunting): picked mon.noname-c con 0x7f794813e320 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:42.583 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:42.583 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79481d8940 con 0x7f79480b1c40 2023-10-06T13:52:42.583 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:42.583 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79481d8b90 con 0x7f79480b3a00 2023-10-06T13:52:42.584 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:42.584 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79480aa200 con 0x7f794813e320 2023-10-06T13:52:42.584 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:42.584 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.574+0000 7f794daa4640 10 monclient(hunting): authenticate will time out at 1290.761085s 2023-10-06T13:52:42.584 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f79477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79480b1c40 0x7f79481403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:42.584 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f7946ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79480b3a00 0x7f79481d90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:42.584 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f7947fff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f794813e320 0x7f79481382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:42.584 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f7946ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79480b3a00 0x7f79481d90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:42034/0 (socket says 172.21.15.19:42034) 2023-10-06T13:52:42.584 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f7946ffd640 1 -- 172.21.15.19:0/2804379027 learned_addr learned my addr 172.21.15.19:0/2804379027 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:52:42.584 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f7946ffd640 10 monclient(hunting): get_auth_request con 0x7f79480b3a00 auth_method 0 2023-10-06T13:52:42.584 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f7946ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:42.584 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f7946ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:42.585 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f7946ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:42.585 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f7947fff640 10 monclient(hunting): get_auth_request con 0x7f794813e320 auth_method 0 2023-10-06T13:52:42.585 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f7947fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:42.585 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f7947fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:42.585 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f7947fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:42.585 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f79477fe640 10 monclient(hunting): get_auth_request con 0x7f79480b1c40 auth_method 0 2023-10-06T13:52:42.585 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f79477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:42.585 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f79477fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:42.585 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.575+0000 7f79477fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:42.585 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.576+0000 7f7947fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:42.585 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.576+0000 7f7947fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:42.586 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.576+0000 7f7947fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:42.586 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.576+0000 7f7946ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:42.586 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.576+0000 7f7946ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:42.586 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.576+0000 7f7946ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:42.586 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.576+0000 7f79477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:42.586 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.576+0000 7f79477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:42.586 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.576+0000 7f79477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:42.586 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.580+0000 7f7946ffd640 10 monclient(hunting): handle_auth_done global_id 4216 payload 824 2023-10-06T13:52:42.586 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.580+0000 7f7946ffd640 1 -- 172.21.15.19:0/2804379027 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f794813e320 msgr2=0x7f79481382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:42.586 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.580+0000 7f7946ffd640 1 --2- 172.21.15.19:0/2804379027 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f794813e320 0x7f79481382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:42.586 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.580+0000 7f7946ffd640 1 -- 172.21.15.19:0/2804379027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79480b1c40 msgr2=0x7f79481403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:42.587 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.580+0000 7f7946ffd640 1 --2- 172.21.15.19:0/2804379027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79480b1c40 0x7f79481403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:42.587 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.580+0000 7f7946ffd640 10 monclient: _finish_hunting 0 2023-10-06T13:52:42.587 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.580+0000 7f7946ffd640 1 monclient: found mon.noname-b 2023-10-06T13:52:42.587 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.580+0000 7f7946ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:42.587 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.580+0000 7f7946ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:52:42.587 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.580+0000 7f7946ffd640 1 -- 172.21.15.19:0/2804379027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79380094d0 con 0x7f79480b3a00 2023-10-06T13:52:42.587 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.583+0000 7f7946ffd640 1 --2- 172.21.15.19:0/2804379027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79480b3a00 0x7f79481d90b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f793c00e870 tx=0x7f793c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:42.588 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.583+0000 7f792ffff640 10 client.?.objecter ms_handle_connect 0x7f79480b3a00 2023-10-06T13:52:42.588 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.583+0000 7f792ffff640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:42.589 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f792ffff640 1 -- 172.21.15.19:0/2804379027 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c0094f0 con 0x7f79480b3a00 2023-10-06T13:52:42.590 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f792ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:42.590 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f792ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:52:42.590 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f792ffff640 10 monclient: dump: 2023-10-06T13:52:42.590 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:42.590 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:42.590 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:42.590 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:42.590 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:42.590 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:42.590 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:42.590 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:42.591 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:42.591 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:42.591 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f792ffff640 10 monclient: _finish_auth 0 2023-10-06T13:52:42.591 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f792ffff640 10 monclient: _check_auth_tickets 2023-10-06T13:52:42.591 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f792ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:42.591 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f794daa4640 5 monclient: authenticate success, global_id 4216 2023-10-06T13:52:42.591 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f794daa4640 10 monclient: _renew_subs 2023-10-06T13:52:42.591 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f794daa4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:42.591 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f794daa4640 1 -- 172.21.15.19:0/2804379027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79481d95f0 con 0x7f79480b3a00 2023-10-06T13:52:42.591 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f794daa4640 10 client.4216.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:42.591 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f794daa4640 10 monclient: _renew_subs 2023-10-06T13:52:42.591 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f794daa4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:42.592 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f794daa4640 1 -- 172.21.15.19:0/2804379027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79481389f0 con 0x7f79480b3a00 2023-10-06T13:52:42.592 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.587+0000 7f794daa4640 1 librados: init done 2023-10-06T13:52:42.592 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.588+0000 7f792ffff640 1 -- 172.21.15.19:0/2804379027 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c005990 con 0x7f79480b3a00 2023-10-06T13:52:42.592 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.588+0000 7f792ffff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:42.592 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.588+0000 7f7945ffb640 4 set_mon_vals no callback set 2023-10-06T13:52:42.593 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.588+0000 7f792ffff640 1 -- 172.21.15.19:0/2804379027 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c005d00 con 0x7f79480b3a00 2023-10-06T13:52:42.593 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.588+0000 7f792ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:42.593 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.588+0000 7f792ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:42.593 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.588+0000 7f792ffff640 10 monclient: dump: 2023-10-06T13:52:42.593 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:42.593 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:42.593 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:42.593 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:42.593 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:42.593 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:42.593 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:42.593 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:42.594 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:42.594 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:42.594 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.588+0000 7f794daa4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:42.594 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.589+0000 7f794daa4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:42.594 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.589+0000 7f794daa4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:42.594 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.589+0000 7f794daa4640 1 -- 172.21.15.19:0/2804379027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7928005040 con 0x7f79480b3a00 2023-10-06T13:52:42.640 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:17.443 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:42.915 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:17.726 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:42.950 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.948+0000 7f792ffff640 1 -- 172.21.15.19:0/2804379027 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 88859+0+0 (secure 0 0 0) 0x7f793c00c940 con 0x7f79480b3a00 2023-10-06T13:52:42.951 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.949+0000 7f792ffff640 1 --2- 172.21.15.19:0/2804379027 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f793406c130 0x7f793406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:42.951 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.949+0000 7f79477fe640 1 --2- 172.21.15.19:0/2804379027 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f793406c130 0x7f793406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:42.951 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.949+0000 7f79477fe640 10 monclient: get_auth_request con 0x7f793406c130 auth_method 0 2023-10-06T13:52:42.951 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.950+0000 7f79477fe640 1 --2- 172.21.15.19:0/2804379027 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f793406c130 0x7f793406e5e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7938036000 tx=0x7f793803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:42.951 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.950+0000 7f792ffff640 10 client.4216.objecter ms_handle_connect 0x7f793406c130 2023-10-06T13:52:42.971 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.968+0000 7f792ffff640 1 -- 172.21.15.19:0/2804379027 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f793c0583c0 con 0x7f79480b3a00 2023-10-06T13:52:42.971 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.968+0000 7f792ffff640 10 client.4216.objecter ms_dispatch 0x7f7948000b90 osd_map(10..10 src has 1..10) v4 2023-10-06T13:52:42.971 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.968+0000 7f792ffff640 3 client.4216.objecter handle_osd_map got epochs [10,10] > 0 2023-10-06T13:52:42.971 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.968+0000 7f792ffff640 3 client.4216.objecter handle_osd_map decoding full epoch 10 2023-10-06T13:52:42.971 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.968+0000 7f792ffff640 20 client.4216.objecter dump_active .. 0 homeless 2023-10-06T13:52:42.971 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.969+0000 7f792ffff640 1 -- 172.21.15.19:0/2804379027 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f793c054700 con 0x7f79480b3a00 2023-10-06T13:52:42.971 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.969+0000 7f792ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:42.971 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:42.969+0000 7f792ffff640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:43.102 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:17.881 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:43.314 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.311+0000 7f794daa4640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:43.314 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.311+0000 7f794daa4640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:43.314 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.311+0000 7f794daa4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:43.314 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.311+0000 7f794daa4640 1 -- 172.21.15.19:0/2804379027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79280055c0 con 0x7f79480b3a00 2023-10-06T13:52:43.337 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.335+0000 7f792ffff640 1 -- 172.21.15.19:0/2804379027 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f793c021090 con 0x7f79480b3a00 2023-10-06T13:52:43.337 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.335+0000 7f792ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:43.338 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.335+0000 7f792ffff640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:52:43.338 INFO:teuthology.misc.health.smithi019.stdout: 2023-10-06T13:52:43.338 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":10,"fsid":"c5a90f7c-81ec-43a3-ab47-af6ce5729ec5","created":"2023-10-06T13:52:19.516184+0000","modified":"2023-10-06T13:52:24.916896+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T13:52:24.916896+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b6cb75c5-86fb-48b9-895c-fb1d72bd4746","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ba828cb4-a4f6-4fea-a02e-24d9cf597ea7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"06bcfab5-d2f1-4447-ba44-143398707ef4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ad31d4df-e78c-4e32-bfd4-3fe6e70dd428","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"931b23f8-80c3-4595-8931-2568be479cb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"70dd2f95-93eb-448a-a65d-3a09bdea85a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9797eb84-1b2c-4280-ab13-bdb7b2007dd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T13:52:43.340 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 10 librados: watch_flush enter 2023-10-06T13:52:43.340 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 10 librados: watch_flush exit 2023-10-06T13:52:43.340 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 20 client.4216.objecter shutdown clearing up homeless session... 2023-10-06T13:52:43.341 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 10 client.4216.objecter successfully canceled tick 2023-10-06T13:52:43.341 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 1 -- 172.21.15.19:0/2804379027 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f793406c130 msgr2=0x7f793406e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:43.341 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 1 --2- 172.21.15.19:0/2804379027 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f793406c130 0x7f793406e5e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7938036000 tx=0x7f793803c040 comp rx=0 tx=0).stop 2023-10-06T13:52:43.341 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 10 monclient: shutdown 2023-10-06T13:52:43.341 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:43.341 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 1 -- 172.21.15.19:0/2804379027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79480b3a00 msgr2=0x7f79481d90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:43.341 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 1 --2- 172.21.15.19:0/2804379027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79480b3a00 0x7f79481d90b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f793c00e870 tx=0x7f793c00ed30 comp rx=0 tx=0).stop 2023-10-06T13:52:43.341 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 1 -- 172.21.15.19:0/2804379027 shutdown_connections 2023-10-06T13:52:43.341 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 1 --2- 172.21.15.19:0/2804379027 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f793406c130 0x7f793406e5e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:43.341 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 1 --2- 172.21.15.19:0/2804379027 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f794813e320 0x7f79481382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:43.341 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 1 --2- 172.21.15.19:0/2804379027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79480b3a00 0x7f79481d90b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:43.342 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 1 --2- 172.21.15.19:0/2804379027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79480b1c40 0x7f79481403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:43.342 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 1 -- 172.21.15.19:0/2804379027 >> 172.21.15.19:0/2804379027 conn(0x7f79480659b0 msgr2=0x7f79480b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:43.342 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 1 -- 172.21.15.19:0/2804379027 shutdown_connections 2023-10-06T13:52:43.342 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 1 -- 172.21.15.19:0/2804379027 wait complete. 2023-10-06T13:52:43.342 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:43.338+0000 7f794daa4640 1 librados: shutdown 2023-10-06T13:52:43.354 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T13:52:43.741 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:18.539 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:43.747 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:18.539 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:43.810 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:18.601 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:47.852 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:22.660 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:47.858 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:22.666 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:48.287 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:23.089 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:48.291 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:23.093 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:48.320 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:23.116 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:48.326 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:23.122 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:48.634 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:23.445 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:48.638 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:23.450 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:49.001 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:23.780 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:49.005 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:23.784 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:49.448 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:24.240 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:49.452 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:24.244 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:49.508 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:24.306 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:49.512 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:24.310 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:49.601 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:24.392 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:49.608 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:24.398 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:50.160 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T13:52:50.282 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.278+0000 7f29ed2cf640 10 monclient: get_monmap_and_config 2023-10-06T13:52:50.283 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.278+0000 7f29ed2cf640 10 monclient: build_initial_monmap 2023-10-06T13:52:50.283 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.278+0000 7f29ed2cf640 10 monclient: monmap: 2023-10-06T13:52:50.283 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-10-06T13:52:50.283 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:50.283 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:52:50.279424+0000 2023-10-06T13:52:50.283 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:52:50.279424+0000 2023-10-06T13:52:50.283 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:50.284 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:50.284 INFO:teuthology.misc.health.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:52:50.284 INFO:teuthology.misc.health.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:52:50.284 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:52:50.284 INFO:teuthology.misc.health.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:52:50.284 INFO:teuthology.misc.health.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:52:50.284 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:50.285 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.279+0000 7f29ed2cf640 1 Processor -- start 2023-10-06T13:52:50.285 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.279+0000 7f29ed2cf640 1 -- start start 2023-10-06T13:52:50.285 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.279+0000 7f29ed2cf640 10 monclient: init 2023-10-06T13:52:50.285 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.279+0000 7f29ed2cf640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:50.285 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.279+0000 7f29ed2cf640 10 monclient: _add_conns ranks=[1,0,4,2,3] 2023-10-06T13:52:50.285 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.279+0000 7f29ed2cf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f29e80b3310 addr v1:172.21.15.83:6789/0 2023-10-06T13:52:50.285 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.279+0000 7f29ed2cf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f29e80a8950 addr v1:172.21.15.19:6789/0 2023-10-06T13:52:50.285 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.280+0000 7f29ed2cf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f29e806b860 0x7f29e813d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:50.286 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.280+0000 7f29ed2cf640 10 monclient(hunting): picked mon.noname-c con 0x7f29e806b860 addr v2:172.21.15.83:3300/0 2023-10-06T13:52:50.286 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.280+0000 7f29ed2cf640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e80b36e0 con 0x7f29e80a8950 2023-10-06T13:52:50.286 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.280+0000 7f29ed2cf640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e80a8e30 con 0x7f29e80b3310 2023-10-06T13:52:50.286 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.280+0000 7f29ed2cf640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:50.286 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.280+0000 7f29ed2cf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e80a9630 con 0x7f29e806b860 2023-10-06T13:52:50.286 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.280+0000 7f29ed2cf640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:50.286 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.280+0000 7f29ed2cf640 10 monclient(hunting): authenticate will time out at 1298.466476s 2023-10-06T13:52:50.286 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.280+0000 7f29e67fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f29e80a8950 0x7f29e81453a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49804/0 (socket says 172.21.15.19:49804) 2023-10-06T13:52:50.287 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.280+0000 7f29e67fc640 1 -- 172.21.15.19:0/2637885164 learned_addr learned my addr 172.21.15.19:0/2637885164 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:52:50.288 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.283+0000 7f29e77fe640 1 --2- 172.21.15.19:0/2637885164 >> v2:172.21.15.83:3300/0 conn(0x7f29e806b860 0x7f29e813d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:50.288 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.283+0000 7f29e5ffb640 1 -- 172.21.15.19:0/2637885164 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762673042 0 0) 0x7f29e80b36e0 con 0x7f29e80a8950 2023-10-06T13:52:50.288 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.283+0000 7f29e5ffb640 10 monclient(hunting): _negotiate 2023-10-06T13:52:50.288 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.283+0000 7f29e5ffb640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:50.288 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.283+0000 7f29e5ffb640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:50.288 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.283+0000 7f29e5ffb640 10 monclient(hunting): my global_id is 4209 2023-10-06T13:52:50.289 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.283+0000 7f29e5ffb640 1 -- 172.21.15.19:0/2637885164 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29d0003520 con 0x7f29e80a8950 2023-10-06T13:52:50.289 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.283+0000 7f29e77fe640 10 monclient(hunting): get_auth_request con 0x7f29e806b860 auth_method 0 2023-10-06T13:52:50.289 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.284+0000 7f29e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:50.289 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.284+0000 7f29e77fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:50.289 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.284+0000 7f29e77fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:50.289 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.284+0000 7f29e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:50.289 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.284+0000 7f29e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:50.289 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.284+0000 7f29e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:50.291 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.286+0000 7f29e77fe640 10 monclient(hunting): handle_auth_done global_id 4222 payload 274 2023-10-06T13:52:50.291 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.286+0000 7f29e77fe640 1 -- 172.21.15.19:0/2637885164 >> v1:172.21.15.83:6789/0 conn(0x7f29e80b3310 legacy=0x7f29e8071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:50.291 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.287+0000 7f29e77fe640 1 -- 172.21.15.19:0/2637885164 >> v1:172.21.15.19:6789/0 conn(0x7f29e80a8950 legacy=0x7f29e81453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:50.291 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.287+0000 7f29e77fe640 10 monclient: _finish_hunting 0 2023-10-06T13:52:50.291 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.287+0000 7f29e77fe640 1 monclient: found mon.noname-c 2023-10-06T13:52:50.291 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.287+0000 7f29e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:50.291 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.287+0000 7f29e77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:52:50.291 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.287+0000 7f29e77fe640 1 -- 172.21.15.19:0/2637885164 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e80aa810 con 0x7f29e806b860 2023-10-06T13:52:50.292 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.290+0000 7f29e77fe640 1 --2- 172.21.15.19:0/2637885164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e806b860 0x7f29e813d690 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f29e0009d90 tx=0x7f29e002bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1491f034a01aa36d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:50.305 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29e5ffb640 1 -- 172.21.15.19:0/2637885164 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e002ccf0 con 0x7f29e806b860 2023-10-06T13:52:50.305 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:50.306 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29e5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:52:50.306 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29e5ffb640 10 monclient: dump: 2023-10-06T13:52:50.306 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:50.306 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:50.306 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:50.306 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:50.307 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:50.307 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:50.307 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:50.307 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:50.307 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:50.307 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:50.308 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29e5ffb640 10 monclient: _finish_auth 0 2023-10-06T13:52:50.308 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29e5ffb640 10 monclient: _check_auth_tickets 2023-10-06T13:52:50.308 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:50.308 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29e5ffb640 1 -- 172.21.15.19:0/2637885164 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e002ce50 con 0x7f29e806b860 2023-10-06T13:52:50.308 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29e5ffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:50.308 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29e5ffb640 1 -- 172.21.15.19:0/2637885164 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e0034660 con 0x7f29e806b860 2023-10-06T13:52:50.308 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:50.309 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29e5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:50.309 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29e5ffb640 10 monclient: dump: 2023-10-06T13:52:50.309 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:50.309 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:50.309 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:50.309 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:50.309 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:50.310 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:50.310 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:50.310 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:50.310 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:50.310 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:50.310 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29ed2cf640 5 monclient: authenticate success, global_id 4222 2023-10-06T13:52:50.310 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29ed2cf640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:50.311 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.302+0000 7f29ed2cf640 4 set_mon_vals no callback set 2023-10-06T13:52:50.311 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.303+0000 7f29ed2cf640 10 monclient: shutdown 2023-10-06T13:52:50.311 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.303+0000 7f29ed2cf640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:50.311 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.303+0000 7f29ed2cf640 1 -- 172.21.15.19:0/2637885164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e806b860 msgr2=0x7f29e813d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:50.311 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.303+0000 7f29ed2cf640 1 --2- 172.21.15.19:0/2637885164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e806b860 0x7f29e813d690 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f29e0009d90 tx=0x7f29e002bf40 comp rx=0 tx=0).stop 2023-10-06T13:52:50.313 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.303+0000 7f29ed2cf640 1 -- 172.21.15.19:0/2637885164 shutdown_connections 2023-10-06T13:52:50.313 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.303+0000 7f29ed2cf640 1 --2- 172.21.15.19:0/2637885164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e806b860 0x7f29e813d690 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:50.313 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.303+0000 7f29ed2cf640 1 -- 172.21.15.19:0/2637885164 >> 172.21.15.19:0/2637885164 conn(0x7f29e8065da0 msgr2=0x7f29e80661a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:50.313 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.303+0000 7f29ed2cf640 1 -- 172.21.15.19:0/2637885164 shutdown_connections 2023-10-06T13:52:50.314 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.304+0000 7f29ed2cf640 1 -- 172.21.15.19:0/2637885164 wait complete. 2023-10-06T13:52:50.319 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.304+0000 7f29ed2cf640 10 monclient: build_initial_monmap 2023-10-06T13:52:50.319 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.304+0000 7f29ed2cf640 10 monclient: monmap: 2023-10-06T13:52:50.319 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-10-06T13:52:50.319 INFO:teuthology.misc.health.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:50.319 INFO:teuthology.misc.health.smithi019.stderr:last_changed 0.000000 2023-10-06T13:52:50.319 INFO:teuthology.misc.health.smithi019.stderr:created 0.000000 2023-10-06T13:52:50.320 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:50.320 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:50.320 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:50.320 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:50.320 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:50.320 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:50.320 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.305+0000 7f29ed2cf640 1 librados: starting msgr at 2023-10-06T13:52:50.320 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.305+0000 7f29ed2cf640 1 librados: starting objecter 2023-10-06T13:52:50.320 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.305+0000 7f29ed2cf640 1 Processor -- start 2023-10-06T13:52:50.320 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.305+0000 7f29ed2cf640 1 -- start start 2023-10-06T13:52:50.320 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.305+0000 7f29ed2cf640 1 librados: setting wanted keys 2023-10-06T13:52:50.320 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.305+0000 7f29ed2cf640 1 librados: calling monclient init 2023-10-06T13:52:50.320 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.305+0000 7f29ed2cf640 10 monclient: init 2023-10-06T13:52:50.321 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:50.321 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 10 monclient: _add_conns ranks=[2,1,0] 2023-10-06T13:52:50.321 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f29e806b860 0x7f29e80701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:50.321 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 10 monclient(hunting): picked mon.noname-c con 0x7f29e806b860 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:50.321 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e80a8950 0x7f29e813fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:50.321 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 10 monclient(hunting): picked mon.noname-b con 0x7f29e80a8950 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:50.321 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e80b3310 0x7f29e813bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:50.321 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 10 monclient(hunting): picked mon.noname-a con 0x7f29e80b3310 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:50.321 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:50.321 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e813f040 con 0x7f29e80b3310 2023-10-06T13:52:50.321 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:50.322 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e806f210 con 0x7f29e80a8950 2023-10-06T13:52:50.322 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:50.322 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e81d4ab0 con 0x7f29e806b860 2023-10-06T13:52:50.322 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:50.322 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29ed2cf640 10 monclient(hunting): authenticate will time out at 1298.492673s 2023-10-06T13:52:50.322 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.306+0000 7f29e77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e80b3310 0x7f29e813bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:50.322 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.307+0000 7f29e77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e80b3310 0x7f29e813bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34218/0 (socket says 172.21.15.19:34218) 2023-10-06T13:52:50.322 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.307+0000 7f29e77fe640 1 -- 172.21.15.19:0/3380971045 learned_addr learned my addr 172.21.15.19:0/3380971045 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:52:50.322 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.307+0000 7f29e77fe640 10 monclient(hunting): get_auth_request con 0x7f29e80b3310 auth_method 0 2023-10-06T13:52:50.322 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.307+0000 7f29e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:50.322 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.307+0000 7f29e77fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:50.323 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.307+0000 7f29e77fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:50.323 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.307+0000 7f29e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:50.323 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.307+0000 7f29e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:50.323 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.307+0000 7f29e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:50.323 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.308+0000 7f29e6ffd640 1 --2- 172.21.15.19:0/3380971045 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f29e806b860 0x7f29e80701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:50.323 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.308+0000 7f29e6ffd640 10 monclient(hunting): get_auth_request con 0x7f29e806b860 auth_method 0 2023-10-06T13:52:50.323 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.308+0000 7f29e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:50.323 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.308+0000 7f29e6ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:50.323 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.308+0000 7f29e6ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:50.323 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.310+0000 7f29e67fc640 1 --2- 172.21.15.19:0/3380971045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e80a8950 0x7f29e813fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:50.323 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.311+0000 7f29e67fc640 10 monclient(hunting): get_auth_request con 0x7f29e80a8950 auth_method 0 2023-10-06T13:52:50.323 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.311+0000 7f29e67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:50.323 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.311+0000 7f29e67fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:50.324 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.311+0000 7f29e67fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:50.324 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.311+0000 7f29e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:50.324 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.311+0000 7f29e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:50.324 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.311+0000 7f29e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:50.324 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.312+0000 7f29e67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:50.324 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.312+0000 7f29e67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:50.324 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.312+0000 7f29e67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:50.324 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.314+0000 7f29e6ffd640 10 monclient(hunting): handle_auth_done global_id 4172 payload 824 2023-10-06T13:52:50.324 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.314+0000 7f29e6ffd640 1 -- 172.21.15.19:0/3380971045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e80a8950 msgr2=0x7f29e813fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:50.324 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.314+0000 7f29e6ffd640 1 --2- 172.21.15.19:0/3380971045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e80a8950 0x7f29e813fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:50.324 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.314+0000 7f29e6ffd640 1 -- 172.21.15.19:0/3380971045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e80b3310 msgr2=0x7f29e813bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:50.324 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.314+0000 7f29e6ffd640 1 --2- 172.21.15.19:0/3380971045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e80b3310 0x7f29e813bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:50.324 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.314+0000 7f29e6ffd640 10 monclient: _finish_hunting 0 2023-10-06T13:52:50.325 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.314+0000 7f29e6ffd640 1 monclient: found mon.noname-c 2023-10-06T13:52:50.325 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.314+0000 7f29e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:50.325 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.314+0000 7f29e6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T13:52:50.325 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.314+0000 7f29e6ffd640 1 -- 172.21.15.19:0/3380971045 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e0009a70 con 0x7f29e806b860 2023-10-06T13:52:50.325 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.317+0000 7f29e6ffd640 1 --2- 172.21.15.19:0/3380971045 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f29e806b860 0x7f29e80701d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c390 tx=0x7f29d800c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:50.325 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.317+0000 7f29cf7fe640 10 client.?.objecter ms_handle_connect 0x7f29e806b860 2023-10-06T13:52:50.325 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.317+0000 7f29cf7fe640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:50.374 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.370+0000 7f29cf7fe640 1 -- 172.21.15.19:0/3380971045 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80052d0 con 0x7f29e806b860 2023-10-06T13:52:50.374 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:50.374 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29cf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:52:50.374 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29cf7fe640 10 monclient: dump: 2023-10-06T13:52:50.374 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:50.374 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:50.374 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:50.374 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:50.374 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:50.374 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:50.375 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:50.375 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:50.375 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:50.375 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:50.375 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29cf7fe640 10 monclient: _finish_auth 0 2023-10-06T13:52:50.375 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29cf7fe640 10 monclient: _check_auth_tickets 2023-10-06T13:52:50.375 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:50.375 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29cf7fe640 1 -- 172.21.15.19:0/3380971045 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d8005430 con 0x7f29e806b860 2023-10-06T13:52:50.375 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29cf7fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:50.375 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29cf7fe640 1 -- 172.21.15.19:0/3380971045 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8003830 con 0x7f29e806b860 2023-10-06T13:52:50.375 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:50.375 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29cf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:52:50.376 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29cf7fe640 10 monclient: dump: 2023-10-06T13:52:50.376 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:50.376 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:50.376 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:50.376 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:50.376 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:50.377 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:50.377 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:50.377 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:50.377 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:50.377 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:50.378 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29ed2cf640 5 monclient: authenticate success, global_id 4172 2023-10-06T13:52:50.378 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29ed2cf640 10 monclient: _renew_subs 2023-10-06T13:52:50.380 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29ed2cf640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:52:50.380 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29ed2cf640 1 -- 172.21.15.19:0/3380971045 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e81d4d30 con 0x7f29e806b860 2023-10-06T13:52:50.380 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29ed2cf640 10 client.4172.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:50.380 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29ed2cf640 10 monclient: _renew_subs 2023-10-06T13:52:50.381 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29ed2cf640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:52:50.381 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29ed2cf640 1 -- 172.21.15.19:0/3380971045 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e806f910 con 0x7f29e806b860 2023-10-06T13:52:50.381 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.371+0000 7f29ed2cf640 1 librados: init done 2023-10-06T13:52:50.381 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.372+0000 7f29ed2cf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:50.381 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.372+0000 7f29ed2cf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:50.381 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.372+0000 7f29ed2cf640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:52:50.381 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.372+0000 7f29ed2cf640 1 -- 172.21.15.19:0/3380971045 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e806f070 con 0x7f29e806b860 2023-10-06T13:52:50.381 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.372+0000 7f29e57fa640 4 set_mon_vals no callback set 2023-10-06T13:52:50.407 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.405+0000 7f29cf7fe640 1 -- 172.21.15.19:0/3380971045 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 4) v1 ==== 88859+0+0 (secure 0 0 0) 0x7f29d8003990 con 0x7f29e806b860 2023-10-06T13:52:50.407 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.406+0000 7f29cf7fe640 1 --2- 172.21.15.19:0/3380971045 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f29d406c180 0x7f29d406e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:50.408 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.406+0000 7f29e67fc640 1 --2- 172.21.15.19:0/3380971045 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f29d406c180 0x7f29d406e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:50.408 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.406+0000 7f29e67fc640 10 monclient: get_auth_request con 0x7f29d406c180 auth_method 0 2023-10-06T13:52:50.408 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.407+0000 7f29cf7fe640 10 client.4172.objecter ms_handle_connect 0x7f29d406c180 2023-10-06T13:52:50.409 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.407+0000 7f29e67fc640 1 --2- 172.21.15.19:0/3380971045 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f29d406c180 0x7f29d406e630 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f29e806e8a0 tx=0x7f29dc007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:50.412 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.410+0000 7f29cf7fe640 1 -- 172.21.15.19:0/3380971045 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f29d8057710 con 0x7f29e806b860 2023-10-06T13:52:50.412 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.410+0000 7f29cf7fe640 10 client.4172.objecter ms_dispatch 0x7f29e8000b90 osd_map(10..10 src has 1..10) v4 2023-10-06T13:52:50.412 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.410+0000 7f29cf7fe640 3 client.4172.objecter handle_osd_map got epochs [10,10] > 0 2023-10-06T13:52:50.412 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.410+0000 7f29cf7fe640 3 client.4172.objecter handle_osd_map decoding full epoch 10 2023-10-06T13:52:50.412 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.410+0000 7f29cf7fe640 20 client.4172.objecter dump_active .. 0 homeless 2023-10-06T13:52:50.733 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.731+0000 7f29cf7fe640 1 -- 172.21.15.19:0/3380971045 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f29d8053a50 con 0x7f29e806b860 2023-10-06T13:52:50.733 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.731+0000 7f29cf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:50.733 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:50.731+0000 7f29cf7fe640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:51.023 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.021+0000 7f29ed2cf640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:51.023 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.021+0000 7f29ed2cf640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:51.023 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.021+0000 7f29ed2cf640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:52:51.023 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.021+0000 7f29ed2cf640 1 -- 172.21.15.19:0/3380971045 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29e806f070 con 0x7f29e806b860 2023-10-06T13:52:51.071 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.069+0000 7f29cf7fe640 1 -- 172.21.15.19:0/3380971045 <== mon.2 v2:172.21.15.19:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f29e806f070 con 0x7f29e806b860 2023-10-06T13:52:51.071 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.069+0000 7f29cf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:51.071 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.069+0000 7f29cf7fe640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:52:51.072 INFO:teuthology.misc.health.smithi019.stdout: 2023-10-06T13:52:51.072 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":10,"fsid":"c5a90f7c-81ec-43a3-ab47-af6ce5729ec5","created":"2023-10-06T13:52:19.516184+0000","modified":"2023-10-06T13:52:24.916896+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T13:52:24.916896+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b6cb75c5-86fb-48b9-895c-fb1d72bd4746","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ba828cb4-a4f6-4fea-a02e-24d9cf597ea7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"06bcfab5-d2f1-4447-ba44-143398707ef4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ad31d4df-e78c-4e32-bfd4-3fe6e70dd428","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"931b23f8-80c3-4595-8931-2568be479cb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"70dd2f95-93eb-448a-a65d-3a09bdea85a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9797eb84-1b2c-4280-ab13-bdb7b2007dd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T13:52:51.075 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 10 librados: watch_flush enter 2023-10-06T13:52:51.075 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 10 librados: watch_flush exit 2023-10-06T13:52:51.075 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 20 client.4172.objecter shutdown clearing up homeless session... 2023-10-06T13:52:51.075 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 10 client.4172.objecter successfully canceled tick 2023-10-06T13:52:51.075 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 1 -- 172.21.15.19:0/3380971045 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f29d406c180 msgr2=0x7f29d406e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:51.075 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 1 --2- 172.21.15.19:0/3380971045 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f29d406c180 0x7f29d406e630 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f29e806e8a0 tx=0x7f29dc007d30 comp rx=0 tx=0).stop 2023-10-06T13:52:51.075 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 10 monclient: shutdown 2023-10-06T13:52:51.076 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:51.076 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 1 -- 172.21.15.19:0/3380971045 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f29e806b860 msgr2=0x7f29e80701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:51.076 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 1 --2- 172.21.15.19:0/3380971045 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f29e806b860 0x7f29e80701d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c390 tx=0x7f29d800c850 comp rx=0 tx=0).stop 2023-10-06T13:52:51.076 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 1 -- 172.21.15.19:0/3380971045 shutdown_connections 2023-10-06T13:52:51.076 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 1 --2- 172.21.15.19:0/3380971045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e80b3310 0x7f29e813bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:51.076 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 1 --2- 172.21.15.19:0/3380971045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e80a8950 0x7f29e813fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:51.076 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 1 --2- 172.21.15.19:0/3380971045 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f29d406c180 0x7f29d406e630 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:51.076 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 1 --2- 172.21.15.19:0/3380971045 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f29e806b860 0x7f29e80701d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:51.076 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.072+0000 7f29ed2cf640 1 -- 172.21.15.19:0/3380971045 >> 172.21.15.19:0/3380971045 conn(0x7f29e8065da0 msgr2=0x7f29e813d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:51.076 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.073+0000 7f29ed2cf640 1 -- 172.21.15.19:0/3380971045 shutdown_connections 2023-10-06T13:52:51.076 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.073+0000 7f29ed2cf640 1 -- 172.21.15.19:0/3380971045 wait complete. 2023-10-06T13:52:51.076 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:51.073+0000 7f29ed2cf640 1 librados: shutdown 2023-10-06T13:52:51.088 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T13:52:52.113 INFO:tasks.ceph.osd.6.smithi083.stderr:==00:00:00:26.922 86124== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:52.576 INFO:tasks.ceph.osd.5.smithi083.stderr:==00:00:00:27.372 86126== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:52.645 INFO:tasks.ceph.osd.7.smithi083.stderr:==00:00:00:27.447 86125== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:53.187 INFO:tasks.ceph.osd.4.smithi083.stderr:==00:00:00:27.998 86123== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:53.473 INFO:tasks.ceph.osd.3.smithi019.stderr:==00:00:00:28.252 86196== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:53.735 INFO:tasks.ceph.osd.2.smithi019.stderr:==00:00:00:28.527 86194== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:54.099 INFO:tasks.ceph.osd.1.smithi019.stderr:==00:00:00:28.897 86193== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:54.460 INFO:tasks.ceph.osd.0.smithi019.stderr:==00:00:00:29.250 86195== Warning: unimplemented fcntl command: 1038 2023-10-06T13:52:57.893 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T13:52:57.987 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.984+0000 7f95705a3640 10 monclient: get_monmap_and_config 2023-10-06T13:52:57.987 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.984+0000 7f95705a3640 10 monclient: build_initial_monmap 2023-10-06T13:52:57.987 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.984+0000 7f95705a3640 10 monclient: monmap: 2023-10-06T13:52:57.987 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-10-06T13:52:57.988 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:57.988 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:52:57.985848+0000 2023-10-06T13:52:57.988 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:52:57.985848+0000 2023-10-06T13:52:57.988 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:57.988 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:57.988 INFO:teuthology.misc.health.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:52:57.988 INFO:teuthology.misc.health.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:52:57.988 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:52:57.988 INFO:teuthology.misc.health.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:52:57.988 INFO:teuthology.misc.health.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:52:57.988 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:57.988 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.985+0000 7f95705a3640 1 Processor -- start 2023-10-06T13:52:57.989 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.985+0000 7f95705a3640 1 -- start start 2023-10-06T13:52:57.989 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.985+0000 7f95705a3640 10 monclient: init 2023-10-06T13:52:57.991 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.986+0000 7f95705a3640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:57.991 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.986+0000 7f95705a3640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2023-10-06T13:52:57.991 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.986+0000 7f95705a3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f95680a8450 0x7f9568144130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:57.991 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.986+0000 7f95705a3640 10 monclient(hunting): picked mon.noname-c con 0x7f95680a8450 addr v2:172.21.15.83:3300/0 2023-10-06T13:52:57.991 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.986+0000 7f95705a3640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f95680b3050 0x7f9568144670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:57.991 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.986+0000 7f95705a3640 10 monclient(hunting): picked mon.noname-b con 0x7f95680b3050 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:57.991 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.986+0000 7f95705a3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f95680a8820 addr v1:172.21.15.83:6789/0 2023-10-06T13:52:57.992 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.986+0000 7f95705a3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95680a98b0 con 0x7f95680a8820 2023-10-06T13:52:57.992 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.986+0000 7f95705a3640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:57.992 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.986+0000 7f95705a3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95680a9b90 con 0x7f95680a8450 2023-10-06T13:52:57.992 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.986+0000 7f95705a3640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:57.992 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.986+0000 7f95705a3640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95680b2850 con 0x7f95680b3050 2023-10-06T13:52:57.992 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.986+0000 7f95705a3640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:57.992 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.986+0000 7f95705a3640 10 monclient(hunting): authenticate will time out at 1306.173156s 2023-10-06T13:52:57.992 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.987+0000 7f956e33f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f95680a8450 0x7f9568144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:57.992 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.987+0000 7f956db3e640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f95680b3050 0x7f9568144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:57.992 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.987+0000 7f956db3e640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f95680b3050 0x7f9568144670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:54264/0 (socket says 172.21.15.19:54264) 2023-10-06T13:52:57.992 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.987+0000 7f956db3e640 1 -- 172.21.15.19:0/2558388683 learned_addr learned my addr 172.21.15.19:0/2558388683 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:52:57.993 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.987+0000 7f956db3e640 10 monclient(hunting): get_auth_request con 0x7f95680b3050 auth_method 0 2023-10-06T13:52:57.993 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.987+0000 7f956db3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:57.993 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.987+0000 7f956db3e640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:57.993 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.987+0000 7f956db3e640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:57.993 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.987+0000 7f956e33f640 10 monclient(hunting): get_auth_request con 0x7f95680a8450 auth_method 0 2023-10-06T13:52:57.993 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.987+0000 7f956e33f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:57.993 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.987+0000 7f956e33f640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:57.993 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.987+0000 7f956e33f640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:57.993 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.988+0000 7f956db3e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:57.993 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.988+0000 7f956db3e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:57.993 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.988+0000 7f956db3e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:57.993 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.988+0000 7f956e33f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:57.994 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.988+0000 7f956e33f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:57.994 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.988+0000 7f956e33f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:57.994 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.990+0000 7f956db3e640 10 monclient(hunting): handle_auth_done global_id 4175 payload 274 2023-10-06T13:52:57.994 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.990+0000 7f956db3e640 1 -- 172.21.15.19:0/2558388683 >> v2:172.21.15.83:3300/0 conn(0x7f95680a8450 msgr2=0x7f9568144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:57.994 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.990+0000 7f956db3e640 1 --2- 172.21.15.19:0/2558388683 >> v2:172.21.15.83:3300/0 conn(0x7f95680a8450 0x7f9568144130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:57.994 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.990+0000 7f956db3e640 1 -- 172.21.15.19:0/2558388683 >> v1:172.21.15.83:6789/0 conn(0x7f95680a8820 legacy=0x7f95680b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:52:57.995 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.990+0000 7f956db3e640 10 monclient: _finish_hunting 0 2023-10-06T13:52:57.995 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.990+0000 7f956db3e640 1 monclient: found mon.noname-b 2023-10-06T13:52:57.995 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.990+0000 7f956db3e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:57.995 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.990+0000 7f956db3e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T13:52:57.995 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.990+0000 7f956db3e640 1 -- 172.21.15.19:0/2558388683 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95680b2b30 con 0x7f95680b3050 2023-10-06T13:52:57.995 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.993+0000 7f956db3e640 1 --2- 172.21.15.19:0/2558388683 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f95680b3050 0x7f9568144670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9558009870 tx=0x7f955802b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7669cec8df6862cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:58.002 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.997+0000 7f956d33d640 1 -- 172.21.15.19:0/2558388683 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95580317c0 con 0x7f95680b3050 2023-10-06T13:52:58.002 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f956d33d640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:58.002 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f956d33d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:52:58.002 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f956d33d640 10 monclient: dump: 2023-10-06T13:52:58.002 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:58.002 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:58.003 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:58.003 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:58.003 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:58.003 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:58.003 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:58.003 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:58.003 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:58.003 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:58.003 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f956d33d640 10 monclient: _finish_auth 0 2023-10-06T13:52:58.003 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f956d33d640 10 monclient: _check_auth_tickets 2023-10-06T13:52:58.004 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f956d33d640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:58.004 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f956d33d640 1 -- 172.21.15.19:0/2558388683 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9558031920 con 0x7f95680b3050 2023-10-06T13:52:58.004 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f95705a3640 5 monclient: authenticate success, global_id 4175 2023-10-06T13:52:58.004 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f95705a3640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:52:58.004 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f956d33d640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:58.004 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f956d33d640 1 -- 172.21.15.19:0/2558388683 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9558031cb0 con 0x7f95680b3050 2023-10-06T13:52:58.004 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f95705a3640 10 monclient: get_monmap_and_config success 2023-10-06T13:52:58.004 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f95705a3640 4 set_mon_vals no callback set 2023-10-06T13:52:58.005 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f956d33d640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:58.005 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f956d33d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:52:58.005 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.998+0000 7f956d33d640 10 monclient: dump: 2023-10-06T13:52:58.005 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:58.005 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:58.005 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:58.005 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:58.005 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:58.005 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:58.005 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:58.005 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:58.005 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:58.006 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:58.006 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.999+0000 7f95705a3640 10 monclient: shutdown 2023-10-06T13:52:58.006 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.999+0000 7f95705a3640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:58.006 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.999+0000 7f95705a3640 1 -- 172.21.15.19:0/2558388683 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f95680b3050 msgr2=0x7f9568144670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:58.006 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.999+0000 7f95705a3640 1 --2- 172.21.15.19:0/2558388683 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f95680b3050 0x7f9568144670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9558009870 tx=0x7f955802b860 comp rx=0 tx=0).stop 2023-10-06T13:52:58.006 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.999+0000 7f95705a3640 1 -- 172.21.15.19:0/2558388683 shutdown_connections 2023-10-06T13:52:58.006 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.999+0000 7f95705a3640 1 --2- 172.21.15.19:0/2558388683 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f95680b3050 0x7f9568144670 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:58.006 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.999+0000 7f95705a3640 1 --2- 172.21.15.19:0/2558388683 >> v2:172.21.15.83:3300/0 conn(0x7f95680a8450 0x7f9568144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:58.006 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.999+0000 7f95705a3640 1 -- 172.21.15.19:0/2558388683 >> 172.21.15.19:0/2558388683 conn(0x7f9568071490 msgr2=0x7f9568071890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:58.006 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.999+0000 7f95705a3640 1 -- 172.21.15.19:0/2558388683 shutdown_connections 2023-10-06T13:52:58.007 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:57.999+0000 7f95705a3640 1 -- 172.21.15.19:0/2558388683 wait complete. 2023-10-06T13:52:58.007 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.000+0000 7f95705a3640 10 monclient: build_initial_monmap 2023-10-06T13:52:58.007 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.000+0000 7f95705a3640 10 monclient: monmap: 2023-10-06T13:52:58.007 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-10-06T13:52:58.007 INFO:teuthology.misc.health.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:52:58.007 INFO:teuthology.misc.health.smithi019.stderr:last_changed 0.000000 2023-10-06T13:52:58.007 INFO:teuthology.misc.health.smithi019.stderr:created 0.000000 2023-10-06T13:52:58.007 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:52:58.007 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:58.007 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:52:58.007 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:52:58.007 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:52:58.007 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:58.008 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.000+0000 7f95705a3640 1 librados: starting msgr at 2023-10-06T13:52:58.008 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.000+0000 7f95705a3640 1 librados: starting objecter 2023-10-06T13:52:58.008 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.000+0000 7f95705a3640 1 Processor -- start 2023-10-06T13:52:58.011 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 1 -- start start 2023-10-06T13:52:58.011 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 1 librados: setting wanted keys 2023-10-06T13:52:58.011 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 1 librados: calling monclient init 2023-10-06T13:52:58.011 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 10 monclient: init 2023-10-06T13:52:58.012 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 10 monclient: _reopen_session rank -1 2023-10-06T13:52:58.012 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 10 monclient: _add_conns ranks=[2,0,1] 2023-10-06T13:52:58.012 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f95680a8450 0x7f956806c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:58.012 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 10 monclient(hunting): picked mon.noname-c con 0x7f95680a8450 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:52:58.012 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95680a8820 0x7f95681da4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:58.012 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 10 monclient(hunting): picked mon.noname-a con 0x7f95680a8820 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:52:58.012 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95680b3050 0x7f95681daa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:58.012 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 10 monclient(hunting): picked mon.noname-b con 0x7f95680b3050 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:52:58.012 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:58.012 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956813db60 con 0x7f95680a8820 2023-10-06T13:52:58.012 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:58.012 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95681daf40 con 0x7f95680b3050 2023-10-06T13:52:58.013 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 10 monclient(hunting): start opening mon connection 2023-10-06T13:52:58.013 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956813e2d0 con 0x7f95680a8450 2023-10-06T13:52:58.013 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 10 monclient(hunting): _renew_subs 2023-10-06T13:52:58.013 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.001+0000 7f95705a3640 10 monclient(hunting): authenticate will time out at 1306.188118s 2023-10-06T13:52:58.013 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.002+0000 7f956e33f640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f95680a8450 0x7f956806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:58.013 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.002+0000 7f956eb40640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95680b3050 0x7f95681daa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:58.013 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.002+0000 7f956e33f640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f95680a8450 0x7f956806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:54268/0 (socket says 172.21.15.19:54268) 2023-10-06T13:52:58.013 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.002+0000 7f956e33f640 1 -- 172.21.15.19:0/3831673244 learned_addr learned my addr 172.21.15.19:0/3831673244 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:52:58.013 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.002+0000 7f956e33f640 10 monclient(hunting): get_auth_request con 0x7f95680a8450 auth_method 0 2023-10-06T13:52:58.013 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.002+0000 7f956e33f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:58.014 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.002+0000 7f956e33f640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:58.014 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.002+0000 7f956e33f640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:58.014 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956eb40640 10 monclient(hunting): get_auth_request con 0x7f95680b3050 auth_method 0 2023-10-06T13:52:58.014 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956eb40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:58.014 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956eb40640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:58.014 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956eb40640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:58.014 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956db3e640 1 --2- 172.21.15.19:0/3831673244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95680a8820 0x7f95681da4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:58.014 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956e33f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:58.014 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956e33f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:58.014 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956e33f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:58.015 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956eb40640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:58.015 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956eb40640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:58.015 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956eb40640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:58.015 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956db3e640 10 monclient(hunting): get_auth_request con 0x7f95680a8820 auth_method 0 2023-10-06T13:52:58.015 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956db3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:52:58.015 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956db3e640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:52:58.015 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.003+0000 7f956db3e640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:52:58.015 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.004+0000 7f956db3e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:52:58.015 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.004+0000 7f956db3e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:52:58.015 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.004+0000 7f956db3e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:52:58.015 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.006+0000 7f956eb40640 10 monclient(hunting): handle_auth_done global_id 4231 payload 824 2023-10-06T13:52:58.016 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.006+0000 7f956eb40640 1 -- 172.21.15.19:0/3831673244 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f95680a8450 msgr2=0x7f956806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:58.016 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.006+0000 7f956eb40640 1 --2- 172.21.15.19:0/3831673244 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f95680a8450 0x7f956806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:58.016 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.006+0000 7f956eb40640 1 -- 172.21.15.19:0/3831673244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95680a8820 msgr2=0x7f95681da4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:58.016 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.007+0000 7f956eb40640 1 --2- 172.21.15.19:0/3831673244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95680a8820 0x7f95681da4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:58.016 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.007+0000 7f956eb40640 10 monclient: _finish_hunting 0 2023-10-06T13:52:58.016 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.007+0000 7f956eb40640 1 monclient: found mon.noname-b 2023-10-06T13:52:58.016 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.007+0000 7f956eb40640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:52:58.016 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.007+0000 7f956eb40640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:52:58.016 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.007+0000 7f956eb40640 1 -- 172.21.15.19:0/3831673244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95580094d0 con 0x7f95680b3050 2023-10-06T13:52:58.016 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.009+0000 7f956eb40640 1 --2- 172.21.15.19:0/3831673244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95680b3050 0x7f95681daa00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f95680a8bf0 tx=0x7f9560009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:58.016 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.009+0000 7f955effd640 10 client.?.objecter ms_handle_connect 0x7f95680b3050 2023-10-06T13:52:58.017 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.009+0000 7f955effd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:52:58.019 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f955effd640 1 -- 172.21.15.19:0/3831673244 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956000ce10 con 0x7f95680b3050 2023-10-06T13:52:58.019 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f955effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:58.019 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f955effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:52:58.019 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f955effd640 10 monclient: dump: 2023-10-06T13:52:58.019 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:58.019 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:58.019 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:58.019 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:58.019 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:58.019 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:58.020 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:58.020 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:58.020 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:58.020 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:58.020 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f955effd640 10 monclient: _finish_auth 0 2023-10-06T13:52:58.020 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f955effd640 10 monclient: _check_auth_tickets 2023-10-06T13:52:58.020 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f955effd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:52:58.020 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f95705a3640 5 monclient: authenticate success, global_id 4231 2023-10-06T13:52:58.020 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f95705a3640 10 monclient: _renew_subs 2023-10-06T13:52:58.020 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f95705a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:58.020 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f95705a3640 1 -- 172.21.15.19:0/3831673244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956806d7c0 con 0x7f95680b3050 2023-10-06T13:52:58.020 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f95705a3640 10 client.4231.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:52:58.021 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f95705a3640 10 monclient: _renew_subs 2023-10-06T13:52:58.021 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f95705a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:58.021 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f95705a3640 1 -- 172.21.15.19:0/3831673244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956813cf20 con 0x7f95680b3050 2023-10-06T13:52:58.021 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.011+0000 7f95705a3640 1 librados: init done 2023-10-06T13:52:58.021 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.012+0000 7f95705a3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:58.021 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.012+0000 7f95705a3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:58.021 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.012+0000 7f95705a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:58.021 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.012+0000 7f95705a3640 1 -- 172.21.15.19:0/3831673244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956813dff0 con 0x7f95680b3050 2023-10-06T13:52:58.039 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.036+0000 7f955effd640 1 -- 172.21.15.19:0/3831673244 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9560009810 con 0x7f95680b3050 2023-10-06T13:52:58.039 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.036+0000 7f955effd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:52:58.039 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.036+0000 7f955effd640 1 -- 172.21.15.19:0/3831673244 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956001e640 con 0x7f95680b3050 2023-10-06T13:52:58.039 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.036+0000 7f955effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:52:58.039 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.036+0000 7f955effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:52:58.039 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.036+0000 7f955effd640 10 monclient: dump: 2023-10-06T13:52:58.039 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:52:58.039 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:52:58.039 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:58.039 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:52:58.040 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:52:58.040 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:52:58.040 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:52:58.040 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:52:58.040 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:52:58.040 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:52:58.040 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.036+0000 7f956cb3c640 4 set_mon_vals no callback set 2023-10-06T13:52:58.041 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.037+0000 7f955effd640 1 -- 172.21.15.19:0/3831673244 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 88859+0+0 (secure 0 0 0) 0x7f956001d3d0 con 0x7f95680b3050 2023-10-06T13:52:58.041 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.038+0000 7f955effd640 1 --2- 172.21.15.19:0/3831673244 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f954c06c130 0x7f954c06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:52:58.041 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.038+0000 7f956e33f640 1 --2- 172.21.15.19:0/3831673244 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f954c06c130 0x7f954c06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:52:58.041 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.038+0000 7f956e33f640 10 monclient: get_auth_request con 0x7f954c06c130 auth_method 0 2023-10-06T13:52:58.041 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.039+0000 7f955effd640 10 client.4231.objecter ms_handle_connect 0x7f954c06c130 2023-10-06T13:52:58.041 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.039+0000 7f956e33f640 1 --2- 172.21.15.19:0/3831673244 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f954c06c130 0x7f954c06e5e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9564009b50 tx=0x7f9564009110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:52:58.042 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.040+0000 7f955effd640 1 -- 172.21.15.19:0/3831673244 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f9560058400 con 0x7f95680b3050 2023-10-06T13:52:58.042 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.040+0000 7f955effd640 10 client.4231.objecter ms_dispatch 0x7f9568000b90 osd_map(10..10 src has 1..10) v4 2023-10-06T13:52:58.042 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.040+0000 7f955effd640 3 client.4231.objecter handle_osd_map got epochs [10,10] > 0 2023-10-06T13:52:58.042 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.040+0000 7f955effd640 3 client.4231.objecter handle_osd_map decoding full epoch 10 2023-10-06T13:52:58.042 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.040+0000 7f955effd640 20 client.4231.objecter dump_active .. 0 homeless 2023-10-06T13:52:58.445 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.443+0000 7f955effd640 1 -- 172.21.15.19:0/3831673244 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f95600581d0 con 0x7f95680b3050 2023-10-06T13:52:58.445 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.443+0000 7f955effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:52:58.445 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.443+0000 7f955effd640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:52:58.679 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.677+0000 7f95705a3640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:58.679 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.677+0000 7f95705a3640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:58.680 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.677+0000 7f95705a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:52:58.680 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.677+0000 7f95705a3640 1 -- 172.21.15.19:0/3831673244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f956813dff0 con 0x7f95680b3050 2023-10-06T13:52:58.726 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.724+0000 7f955effd640 1 -- 172.21.15.19:0/3831673244 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f956813dff0 con 0x7f95680b3050 2023-10-06T13:52:58.727 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.724+0000 7f955effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:52:58.727 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.724+0000 7f955effd640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:52:58.727 INFO:teuthology.misc.health.smithi019.stdout: 2023-10-06T13:52:58.727 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":10,"fsid":"c5a90f7c-81ec-43a3-ab47-af6ce5729ec5","created":"2023-10-06T13:52:19.516184+0000","modified":"2023-10-06T13:52:24.916896+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T13:52:24.916896+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b6cb75c5-86fb-48b9-895c-fb1d72bd4746","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ba828cb4-a4f6-4fea-a02e-24d9cf597ea7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"06bcfab5-d2f1-4447-ba44-143398707ef4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ad31d4df-e78c-4e32-bfd4-3fe6e70dd428","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"931b23f8-80c3-4595-8931-2568be479cb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"70dd2f95-93eb-448a-a65d-3a09bdea85a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9797eb84-1b2c-4280-ab13-bdb7b2007dd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T13:52:58.729 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 10 librados: watch_flush enter 2023-10-06T13:52:58.729 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 10 librados: watch_flush exit 2023-10-06T13:52:58.729 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 20 client.4231.objecter shutdown clearing up homeless session... 2023-10-06T13:52:58.729 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 10 client.4231.objecter successfully canceled tick 2023-10-06T13:52:58.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 1 -- 172.21.15.19:0/3831673244 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f954c06c130 msgr2=0x7f954c06e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:58.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 1 --2- 172.21.15.19:0/3831673244 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f954c06c130 0x7f954c06e5e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9564009b50 tx=0x7f9564009110 comp rx=0 tx=0).stop 2023-10-06T13:52:58.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 10 monclient: shutdown 2023-10-06T13:52:58.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:52:58.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 1 -- 172.21.15.19:0/3831673244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95680b3050 msgr2=0x7f95681daa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:52:58.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 1 --2- 172.21.15.19:0/3831673244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95680b3050 0x7f95681daa00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f95680a8bf0 tx=0x7f9560009ef0 comp rx=0 tx=0).stop 2023-10-06T13:52:58.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 1 -- 172.21.15.19:0/3831673244 shutdown_connections 2023-10-06T13:52:58.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 1 --2- 172.21.15.19:0/3831673244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95680b3050 0x7f95681daa00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:58.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 1 --2- 172.21.15.19:0/3831673244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95680a8820 0x7f95681da4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:58.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 1 --2- 172.21.15.19:0/3831673244 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f954c06c130 0x7f954c06e5e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:58.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 1 --2- 172.21.15.19:0/3831673244 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f95680a8450 0x7f956806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:52:58.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 1 -- 172.21.15.19:0/3831673244 >> 172.21.15.19:0/3831673244 conn(0x7f9568071490 msgr2=0x7f9568144470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:52:58.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 1 -- 172.21.15.19:0/3831673244 shutdown_connections 2023-10-06T13:52:58.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 1 -- 172.21.15.19:0/3831673244 wait complete. 2023-10-06T13:52:58.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:52:58.727+0000 7f95705a3640 1 librados: shutdown 2023-10-06T13:52:58.743 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T13:53:00.421 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T13:53:00.417+0000 5dc2340 -1 osd.6 0 log_to_monitors true 2023-10-06T13:53:00.774 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T13:53:00.770+0000 5dc2340 -1 osd.7 0 log_to_monitors true 2023-10-06T13:53:00.872 INFO:tasks.ceph.osd.5.smithi083.stderr:2023-10-06T13:53:00.868+0000 5dc2340 -1 osd.5 0 log_to_monitors true 2023-10-06T13:53:01.588 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:53:01.584+0000 5dc2340 -1 osd.4 0 log_to_monitors true 2023-10-06T13:53:01.958 INFO:tasks.ceph.osd.3.smithi019.stderr:2023-10-06T13:53:01.952+0000 5dc2340 -1 osd.3 0 log_to_monitors true 2023-10-06T13:53:02.540 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T13:53:02.534+0000 5dc2340 -1 osd.2 0 log_to_monitors true 2023-10-06T13:53:02.835 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T13:53:02.831+0000 5dc2340 -1 osd.0 0 log_to_monitors true 2023-10-06T13:53:02.984 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T13:53:02.979+0000 5dc2340 -1 osd.1 0 log_to_monitors true 2023-10-06T13:53:03.990 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T13:53:03.985+0000 ac7d640 -1 osd.7 0 waiting for initial osdmap 2023-10-06T13:53:04.004 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T13:53:03.999+0000 ac7d640 -1 osd.6 0 waiting for initial osdmap 2023-10-06T13:53:04.043 INFO:tasks.ceph.osd.5.smithi083.stderr:2023-10-06T13:53:04.035+0000 c496640 -1 osd.5 0 waiting for initial osdmap 2023-10-06T13:53:04.397 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T13:53:04.383+0000 12261640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T13:53:04.406 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T13:53:04.391+0000 12a62640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T13:53:04.453 INFO:tasks.ceph.osd.5.smithi083.stderr:2023-10-06T13:53:04.439+0000 12261640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T13:53:05.131 INFO:tasks.ceph.osd.3.smithi019.stderr:2023-10-06T13:53:05.126+0000 ac7d640 -1 osd.3 0 waiting for initial osdmap 2023-10-06T13:53:05.168 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:53:05.162+0000 ac7d640 -1 osd.4 0 waiting for initial osdmap 2023-10-06T13:53:05.550 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T13:53:05.554 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:53:05.540+0000 12261640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T13:53:05.589 INFO:tasks.ceph.osd.3.smithi019.stderr:2023-10-06T13:53:05.574+0000 12261640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T13:53:05.641 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.639+0000 7fefa7f55640 10 monclient: get_monmap_and_config 2023-10-06T13:53:05.641 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.639+0000 7fefa7f55640 10 monclient: build_initial_monmap 2023-10-06T13:53:05.641 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.639+0000 7fefa7f55640 10 monclient: monmap: 2023-10-06T13:53:05.641 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-10-06T13:53:05.641 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:05.642 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:53:05.640437+0000 2023-10-06T13:53:05.642 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:53:05.640437+0000 2023-10-06T13:53:05.642 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:05.642 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:05.642 INFO:teuthology.misc.health.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:05.642 INFO:teuthology.misc.health.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:05.642 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:05.642 INFO:teuthology.misc.health.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:05.642 INFO:teuthology.misc.health.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:05.642 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:53:05.644 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.639+0000 7fefa7f55640 1 Processor -- start 2023-10-06T13:53:05.644 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 1 -- start start 2023-10-06T13:53:05.644 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 10 monclient: init 2023-10-06T13:53:05.644 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:05.644 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 10 monclient: _add_conns ranks=[0,4,2,1,3] 2023-10-06T13:53:05.645 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fefa00b3a00 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:05.645 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fefa013e4f0 0x7fefa00b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:05.645 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 10 monclient(hunting): picked mon.noname-c con 0x7fefa013e4f0 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:05.645 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fefa00b1c40 0x7fefa014b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:05.645 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 10 monclient(hunting): picked mon.noname-b con 0x7fefa00b1c40 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:05.645 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa00b3dd0 con 0x7fefa00b3a00 2023-10-06T13:53:05.645 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:05.645 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa00a93d0 con 0x7fefa013e4f0 2023-10-06T13:53:05.645 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:05.645 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa0144ab0 con 0x7fefa00b1c40 2023-10-06T13:53:05.645 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:05.646 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.640+0000 7fefa7f55640 10 monclient(hunting): authenticate will time out at 1313.827165s 2023-10-06T13:53:05.646 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.641+0000 7fefa5cf1640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fefa00b3a00 0x7fefa013dde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39664/0 (socket says 172.21.15.19:39664) 2023-10-06T13:53:05.646 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.641+0000 7fefa5cf1640 1 -- 172.21.15.19:0/1033833663 learned_addr learned my addr 172.21.15.19:0/1033833663 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:05.646 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.641+0000 7fefa64f2640 1 --2- 172.21.15.19:0/1033833663 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fefa00b1c40 0x7fefa014b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:05.646 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.641+0000 7fefa54f0640 1 --2- 172.21.15.19:0/1033833663 >> v2:172.21.15.83:3300/0 conn(0x7fefa013e4f0 0x7fefa00b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:05.646 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.641+0000 7fefa54f0640 10 monclient(hunting): get_auth_request con 0x7fefa013e4f0 auth_method 0 2023-10-06T13:53:05.646 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.641+0000 7fefa54f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:05.646 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.641+0000 7fefa54f0640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:05.646 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.641+0000 7fefa54f0640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:05.646 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.642+0000 7fefa64f2640 10 monclient(hunting): get_auth_request con 0x7fefa00b1c40 auth_method 0 2023-10-06T13:53:05.646 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.642+0000 7fefa64f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:05.646 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.642+0000 7fefa64f2640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:05.647 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.642+0000 7fefa64f2640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:05.647 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.642+0000 7fefa54f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:05.647 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.642+0000 7fefa54f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:05.647 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.642+0000 7fefa54f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:05.647 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.642+0000 7fefa64f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:05.647 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.642+0000 7fefa64f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:05.647 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.642+0000 7fefa64f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:05.648 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.643+0000 7fefa54f0640 10 monclient(hunting): handle_auth_done global_id 4258 payload 274 2023-10-06T13:53:05.648 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.643+0000 7fefa54f0640 1 -- 172.21.15.19:0/1033833663 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fefa00b1c40 msgr2=0x7fefa014b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:05.648 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.643+0000 7fefa54f0640 1 --2- 172.21.15.19:0/1033833663 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fefa00b1c40 0x7fefa014b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:05.648 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.643+0000 7fefa54f0640 1 -- 172.21.15.19:0/1033833663 >> v1:172.21.15.19:6789/0 conn(0x7fefa00b3a00 legacy=0x7fefa013dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:05.648 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.643+0000 7fefa54f0640 10 monclient: _finish_hunting 0 2023-10-06T13:53:05.648 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.643+0000 7fefa54f0640 1 monclient: found mon.noname-c 2023-10-06T13:53:05.648 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.643+0000 7fefa54f0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:05.648 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.643+0000 7fefa54f0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:53:05.649 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.643+0000 7fefa54f0640 1 -- 172.21.15.19:0/1033833663 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa00a95d0 con 0x7fefa013e4f0 2023-10-06T13:53:05.649 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.645+0000 7fefa54f0640 1 --2- 172.21.15.19:0/1033833663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa013e4f0 0x7fefa00b1700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0097a0 tx=0x7fef9c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=204ff2d3c05f1014 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:05.651 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa4cef640 1 -- 172.21.15.19:0/1033833663 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c031830 con 0x7fefa013e4f0 2023-10-06T13:53:05.651 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa4cef640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:05.651 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa4cef640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:05.651 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa4cef640 10 monclient: dump: 2023-10-06T13:53:05.651 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:53:05.651 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:05.651 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:05.651 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:05.651 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:05.651 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:05.652 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:05.652 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:05.652 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:05.652 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:53:05.652 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa4cef640 10 monclient: _finish_auth 0 2023-10-06T13:53:05.652 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa4cef640 10 monclient: _check_auth_tickets 2023-10-06T13:53:05.652 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa4cef640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:05.652 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa4cef640 1 -- 172.21.15.19:0/1033833663 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c031990 con 0x7fefa013e4f0 2023-10-06T13:53:05.652 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa4cef640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:05.652 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa4cef640 1 -- 172.21.15.19:0/1033833663 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c031d20 con 0x7fefa013e4f0 2023-10-06T13:53:05.652 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa7f55640 5 monclient: authenticate success, global_id 4258 2023-10-06T13:53:05.653 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa7f55640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:05.653 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa4cef640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:05.653 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa7f55640 4 set_mon_vals no callback set 2023-10-06T13:53:05.653 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa4cef640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:05.653 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa4cef640 10 monclient: dump: 2023-10-06T13:53:05.653 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:53:05.653 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:05.653 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:05.653 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:05.653 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:05.653 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:05.653 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:05.654 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:05.654 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:05.654 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:53:05.657 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa7f55640 10 monclient: shutdown 2023-10-06T13:53:05.657 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa7f55640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:05.657 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa7f55640 1 -- 172.21.15.19:0/1033833663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa013e4f0 msgr2=0x7fefa00b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:05.657 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.649+0000 7fefa7f55640 1 --2- 172.21.15.19:0/1033833663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa013e4f0 0x7fefa00b1700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0097a0 tx=0x7fef9c02b8d0 comp rx=0 tx=0).stop 2023-10-06T13:53:05.657 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.650+0000 7fefa7f55640 1 -- 172.21.15.19:0/1033833663 shutdown_connections 2023-10-06T13:53:05.657 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.650+0000 7fefa7f55640 1 --2- 172.21.15.19:0/1033833663 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fefa00b1c40 0x7fefa014b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:05.657 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.650+0000 7fefa7f55640 1 --2- 172.21.15.19:0/1033833663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa013e4f0 0x7fefa00b1700 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:05.657 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.650+0000 7fefa7f55640 1 -- 172.21.15.19:0/1033833663 >> 172.21.15.19:0/1033833663 conn(0x7fefa00659b0 msgr2=0x7fefa0065db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:05.657 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.650+0000 7fefa7f55640 1 -- 172.21.15.19:0/1033833663 shutdown_connections 2023-10-06T13:53:05.657 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.650+0000 7fefa7f55640 1 -- 172.21.15.19:0/1033833663 wait complete. 2023-10-06T13:53:05.657 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.650+0000 7fefa7f55640 10 monclient: build_initial_monmap 2023-10-06T13:53:05.658 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.650+0000 7fefa7f55640 10 monclient: monmap: 2023-10-06T13:53:05.658 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-10-06T13:53:05.658 INFO:teuthology.misc.health.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:05.658 INFO:teuthology.misc.health.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:05.658 INFO:teuthology.misc.health.smithi019.stderr:created 0.000000 2023-10-06T13:53:05.658 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:05.658 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:05.658 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:05.658 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:05.658 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:05.658 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:53:05.659 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 1 librados: starting msgr at 2023-10-06T13:53:05.659 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 1 librados: starting objecter 2023-10-06T13:53:05.659 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 1 Processor -- start 2023-10-06T13:53:05.659 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 1 -- start start 2023-10-06T13:53:05.659 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 1 librados: setting wanted keys 2023-10-06T13:53:05.659 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 1 librados: calling monclient init 2023-10-06T13:53:05.659 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 10 monclient: init 2023-10-06T13:53:05.659 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:05.659 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T13:53:05.659 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa00b1c40 0x7fefa011ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:05.659 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 10 monclient(hunting): picked mon.noname-b con 0x7fefa00b1c40 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:05.659 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa00b3a00 0x7fefa00b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:05.660 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 10 monclient(hunting): picked mon.noname-a con 0x7fefa00b3a00 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:05.660 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fefa013e4f0 0x7fefa01d9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:05.660 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 10 monclient(hunting): picked mon.noname-c con 0x7fefa013e4f0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:05.660 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:05.660 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa01d9160 con 0x7fefa00b3a00 2023-10-06T13:53:05.660 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:05.660 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00b0f10 con 0x7fefa00b1c40 2023-10-06T13:53:05.660 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:05.660 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00af8b0 con 0x7fefa013e4f0 2023-10-06T13:53:05.660 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:05.660 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.651+0000 7fefa7f55640 10 monclient(hunting): authenticate will time out at 1313.838137s 2023-10-06T13:53:05.660 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.652+0000 7fefa64f2640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fefa013e4f0 0x7fefa01d9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:05.661 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.652+0000 7fefa5cf1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa00b1c40 0x7fefa011ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:05.661 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.652+0000 7fefa5cf1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa00b1c40 0x7fefa011ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:56594/0 (socket says 172.21.15.19:56594) 2023-10-06T13:53:05.661 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.652+0000 7fefa5cf1640 1 -- 172.21.15.19:0/1059900160 learned_addr learned my addr 172.21.15.19:0/1059900160 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:05.661 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.652+0000 7fefa5cf1640 10 monclient(hunting): get_auth_request con 0x7fefa00b1c40 auth_method 0 2023-10-06T13:53:05.661 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.652+0000 7fefa5cf1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:05.661 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.652+0000 7fefa5cf1640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:05.661 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.652+0000 7fefa5cf1640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:05.661 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.652+0000 7fefa64f2640 10 monclient(hunting): get_auth_request con 0x7fefa013e4f0 auth_method 0 2023-10-06T13:53:05.661 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.652+0000 7fefa64f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:05.661 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.652+0000 7fefa64f2640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:05.661 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.652+0000 7fefa64f2640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:05.661 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.653+0000 7fefa5cf1640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:05.662 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.653+0000 7fefa5cf1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:05.662 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.653+0000 7fefa5cf1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:05.662 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.653+0000 7fefa64f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:05.662 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.653+0000 7fefa64f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:05.662 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.653+0000 7fefa64f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:05.664 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.655+0000 7fefa5cf1640 10 monclient(hunting): handle_auth_done global_id 4261 payload 824 2023-10-06T13:53:05.664 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.655+0000 7fefa5cf1640 1 -- 172.21.15.19:0/1059900160 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fefa013e4f0 msgr2=0x7fefa01d9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:05.664 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.655+0000 7fefa5cf1640 1 --2- 172.21.15.19:0/1059900160 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fefa013e4f0 0x7fefa01d9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:05.664 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.655+0000 7fefa5cf1640 1 -- 172.21.15.19:0/1059900160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa00b3a00 msgr2=0x7fefa00b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:05.664 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.655+0000 7fefa5cf1640 1 --2- 172.21.15.19:0/1059900160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa00b3a00 0x7fefa00b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:05.664 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.655+0000 7fefa5cf1640 10 monclient: _finish_hunting 0 2023-10-06T13:53:05.664 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.655+0000 7fefa5cf1640 1 monclient: found mon.noname-b 2023-10-06T13:53:05.664 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.655+0000 7fefa5cf1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:05.665 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.655+0000 7fefa5cf1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:05.665 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.655+0000 7fefa5cf1640 1 -- 172.21.15.19:0/1059900160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c009400 con 0x7fefa00b1c40 2023-10-06T13:53:05.665 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.657+0000 7fefa5cf1640 1 --2- 172.21.15.19:0/1059900160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa00b1c40 0x7fefa011ec30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef9000c4a0 tx=0x7fef9000c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:05.665 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.657+0000 7fef967fc640 10 client.?.objecter ms_handle_connect 0x7fefa00b1c40 2023-10-06T13:53:05.665 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.657+0000 7fef967fc640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:05.665 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fef967fc640 1 -- 172.21.15.19:0/1059900160 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90005ce0 con 0x7fefa00b1c40 2023-10-06T13:53:05.665 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fef967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:05.665 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fef967fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:05.665 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fef967fc640 10 monclient: dump: 2023-10-06T13:53:05.665 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:53:05.665 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:05.665 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:05.665 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:05.666 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:05.666 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:05.666 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:05.666 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:05.666 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:05.666 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:53:05.666 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fef967fc640 10 monclient: _finish_auth 0 2023-10-06T13:53:05.666 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fef967fc640 10 monclient: _check_auth_tickets 2023-10-06T13:53:05.666 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fef967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:05.666 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fef967fc640 1 -- 172.21.15.19:0/1059900160 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef90005e40 con 0x7fefa00b1c40 2023-10-06T13:53:05.666 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fef967fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:05.667 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fef967fc640 1 -- 172.21.15.19:0/1059900160 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90007270 con 0x7fefa00b1c40 2023-10-06T13:53:05.667 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fefa7f55640 5 monclient: authenticate success, global_id 4261 2023-10-06T13:53:05.667 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fefa7f55640 10 monclient: _renew_subs 2023-10-06T13:53:05.667 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fefa7f55640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:05.667 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fefa7f55640 1 -- 172.21.15.19:0/1059900160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa00ad9e0 con 0x7fefa00b1c40 2023-10-06T13:53:05.667 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fef97fff640 4 set_mon_vals no callback set 2023-10-06T13:53:05.667 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fef967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:05.667 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fef967fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:05.667 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fef967fc640 10 monclient: dump: 2023-10-06T13:53:05.667 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:53:05.667 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:05.667 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:05.668 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:05.668 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:05.668 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:05.668 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:05.668 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:05.668 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:05.668 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:53:05.668 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fefa7f55640 10 client.4261.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:05.668 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fefa7f55640 10 monclient: _renew_subs 2023-10-06T13:53:05.668 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fefa7f55640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:05.668 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fefa7f55640 1 -- 172.21.15.19:0/1059900160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa01d9da0 con 0x7fefa00b1c40 2023-10-06T13:53:05.668 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.660+0000 7fefa7f55640 1 librados: init done 2023-10-06T13:53:05.668 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.661+0000 7fefa7f55640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:05.669 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.661+0000 7fefa7f55640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:05.669 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.661+0000 7fefa7f55640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:05.669 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.661+0000 7fefa7f55640 1 -- 172.21.15.19:0/1059900160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa0120340 con 0x7fefa00b1c40 2023-10-06T13:53:05.904 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.902+0000 7fef967fc640 1 -- 172.21.15.19:0/1059900160 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 88859+0+0 (secure 0 0 0) 0x7fef90007490 con 0x7fefa00b1c40 2023-10-06T13:53:05.904 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.902+0000 7fef967fc640 1 --2- 172.21.15.19:0/1059900160 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fef8406c180 0x7fef8406e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:05.904 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.903+0000 7fefa54f0640 1 --2- 172.21.15.19:0/1059900160 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fef8406c180 0x7fef8406e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:05.905 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.903+0000 7fefa54f0640 10 monclient: get_auth_request con 0x7fef8406c180 auth_method 0 2023-10-06T13:53:05.905 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.903+0000 7fefa54f0640 1 --2- 172.21.15.19:0/1059900160 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fef8406c180 0x7fef8406e630 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fef9c036000 tx=0x7fef9c030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:05.905 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.903+0000 7fef967fc640 10 client.4261.objecter ms_handle_connect 0x7fef8406c180 2023-10-06T13:53:05.920 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.918+0000 7fef967fc640 1 -- 172.21.15.19:0/1059900160 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2463+0+0 (secure 0 0 0) 0x7fef90057930 con 0x7fefa00b1c40 2023-10-06T13:53:05.920 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.918+0000 7fef967fc640 10 client.4261.objecter ms_dispatch 0x7fefa0000b90 osd_map(13..13 src has 1..13) v4 2023-10-06T13:53:05.920 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.918+0000 7fef967fc640 3 client.4261.objecter handle_osd_map got epochs [13,13] > 0 2023-10-06T13:53:05.920 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.918+0000 7fef967fc640 3 client.4261.objecter handle_osd_map decoding full epoch 13 2023-10-06T13:53:05.920 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.918+0000 7fef967fc640 20 client.4261.objecter dump_active .. 0 homeless 2023-10-06T13:53:05.921 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.919+0000 7fef967fc640 1 -- 172.21.15.19:0/1059900160 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7fef90053c70 con 0x7fefa00b1c40 2023-10-06T13:53:05.921 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.919+0000 7fef967fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:05.921 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:05.919+0000 7fef967fc640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:06.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.131+0000 7fefa7f55640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:53:06.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.131+0000 7fefa7f55640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:53:06.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.131+0000 7fefa7f55640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:06.133 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.131+0000 7fefa7f55640 1 -- 172.21.15.19:0/1059900160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefa01d9ff0 con 0x7fefa00b1c40 2023-10-06T13:53:06.253 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.251+0000 7fef967fc640 1 -- 172.21.15.19:0/1059900160 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8322 (secure 0 0 0) 0x7fef90021090 con 0x7fefa00b1c40 2023-10-06T13:53:06.253 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.251+0000 7fef967fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:53:06.253 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.251+0000 7fef967fc640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:06.254 INFO:teuthology.misc.health.smithi019.stdout: 2023-10-06T13:53:06.254 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":14,"fsid":"c5a90f7c-81ec-43a3-ab47-af6ce5729ec5","created":"2023-10-06T13:52:19.516184+0000","modified":"2023-10-06T13:53:05.762047+0000","last_up_change":"2023-10-06T13:53:05.762047+0000","last_in_change":"2023-10-06T13:52:24.916896+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b6cb75c5-86fb-48b9-895c-fb1d72bd4746","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ba828cb4-a4f6-4fea-a02e-24d9cf597ea7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"06bcfab5-d2f1-4447-ba44-143398707ef4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ad31d4df-e78c-4e32-bfd4-3fe6e70dd428","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"931b23f8-80c3-4595-8931-2568be479cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":86126},{"type":"v1","addr":"172.21.15.83:6819","nonce":86126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":86126},{"type":"v1","addr":"172.21.15.83:6821","nonce":86126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":86126},{"type":"v1","addr":"172.21.15.83:6825","nonce":86126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":86126},{"type":"v1","addr":"172.21.15.83:6823","nonce":86126}]},"public_addr":"172.21.15.83:6819/86126","cluster_addr":"172.21.15.83:6821/86126","heartbeat_back_addr":"172.21.15.83:6825/86126","heartbeat_front_addr":"172.21.15.83:6823/86126","state":["exists","up"]},{"osd":6,"uuid":"70dd2f95-93eb-448a-a65d-3a09bdea85a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":86124},{"type":"v1","addr":"172.21.15.83:6803","nonce":86124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":86124},{"type":"v1","addr":"172.21.15.83:6805","nonce":86124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":86124},{"type":"v1","addr":"172.21.15.83:6809","nonce":86124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":86124},{"type":"v1","addr":"172.21.15.83:6807","nonce":86124}]},"public_addr":"172.21.15.83:6803/86124","cluster_addr":"172.21.15.83:6805/86124","heartbeat_back_addr":"172.21.15.83:6809/86124","heartbeat_front_addr":"172.21.15.83:6807/86124","state":["exists","up"]},{"osd":7,"uuid":"9797eb84-1b2c-4280-ab13-bdb7b2007dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":86125},{"type":"v1","addr":"172.21.15.83:6811","nonce":86125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":86125},{"type":"v1","addr":"172.21.15.83:6813","nonce":86125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":86125},{"type":"v1","addr":"172.21.15.83:6817","nonce":86125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":86125},{"type":"v1","addr":"172.21.15.83:6815","nonce":86125}]},"public_addr":"172.21.15.83:6811/86125","cluster_addr":"172.21.15.83:6813/86125","heartbeat_back_addr":"172.21.15.83:6817/86125","heartbeat_front_addr":"172.21.15.83:6815/86125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T13:53:06.256 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 10 librados: watch_flush enter 2023-10-06T13:53:06.256 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 10 librados: watch_flush exit 2023-10-06T13:53:06.256 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 20 client.4261.objecter shutdown clearing up homeless session... 2023-10-06T13:53:06.256 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 10 client.4261.objecter successfully canceled tick 2023-10-06T13:53:06.256 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 1 -- 172.21.15.19:0/1059900160 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fef8406c180 msgr2=0x7fef8406e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:06.256 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 1 --2- 172.21.15.19:0/1059900160 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fef8406c180 0x7fef8406e630 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fef9c036000 tx=0x7fef9c030000 comp rx=0 tx=0).stop 2023-10-06T13:53:06.256 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 10 monclient: shutdown 2023-10-06T13:53:06.256 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:06.257 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 1 -- 172.21.15.19:0/1059900160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa00b1c40 msgr2=0x7fefa011ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:06.257 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 1 --2- 172.21.15.19:0/1059900160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa00b1c40 0x7fefa011ec30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef9000c4a0 tx=0x7fef9000c960 comp rx=0 tx=0).stop 2023-10-06T13:53:06.257 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 1 -- 172.21.15.19:0/1059900160 shutdown_connections 2023-10-06T13:53:06.257 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 1 --2- 172.21.15.19:0/1059900160 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fef8406c180 0x7fef8406e630 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:06.257 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 1 --2- 172.21.15.19:0/1059900160 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fefa013e4f0 0x7fefa01d9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:06.257 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 1 --2- 172.21.15.19:0/1059900160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa00b3a00 0x7fefa00b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:06.257 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 1 --2- 172.21.15.19:0/1059900160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa00b1c40 0x7fefa011ec30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:06.257 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.253+0000 7fefa7f55640 1 -- 172.21.15.19:0/1059900160 >> 172.21.15.19:0/1059900160 conn(0x7fefa00659b0 msgr2=0x7fefa00b7010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:06.257 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.254+0000 7fefa7f55640 1 -- 172.21.15.19:0/1059900160 shutdown_connections 2023-10-06T13:53:06.257 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.254+0000 7fefa7f55640 1 -- 172.21.15.19:0/1059900160 wait complete. 2023-10-06T13:53:06.257 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:06.254+0000 7fefa7f55640 1 librados: shutdown 2023-10-06T13:53:06.265 DEBUG:teuthology.misc:3 of 8 OSDs are up 2023-10-06T13:53:06.463 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T13:53:06.457+0000 c496640 -1 osd.1 0 waiting for initial osdmap 2023-10-06T13:53:06.497 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T13:53:06.493+0000 ac7d640 -1 osd.2 0 waiting for initial osdmap 2023-10-06T13:53:06.505 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T13:53:06.500+0000 ac7d640 -1 osd.0 0 waiting for initial osdmap 2023-10-06T13:53:06.947 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T13:53:06.930+0000 12a62640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T13:53:06.954 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T13:53:06.937+0000 12261640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T13:53:06.994 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T13:53:06.972+0000 12261640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T13:53:13.072 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T13:53:13.171 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.169+0000 7f0a1081f640 10 monclient: get_monmap_and_config 2023-10-06T13:53:13.171 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.169+0000 7f0a1081f640 10 monclient: build_initial_monmap 2023-10-06T13:53:13.171 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.169+0000 7f0a1081f640 10 monclient: monmap: 2023-10-06T13:53:13.172 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-10-06T13:53:13.172 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:13.172 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:53:13.170692+0000 2023-10-06T13:53:13.172 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:53:13.170692+0000 2023-10-06T13:53:13.172 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:13.172 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:13.172 INFO:teuthology.misc.health.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:13.172 INFO:teuthology.misc.health.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:13.172 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:13.172 INFO:teuthology.misc.health.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:13.173 INFO:teuthology.misc.health.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:13.173 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:53:13.174 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.170+0000 7f0a1081f640 1 Processor -- start 2023-10-06T13:53:13.174 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.170+0000 7f0a1081f640 1 -- start start 2023-10-06T13:53:13.175 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.170+0000 7f0a1081f640 10 monclient: init 2023-10-06T13:53:13.175 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a1081f640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:13.175 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a1081f640 10 monclient: _add_conns ranks=[0,4,3,1,2] 2023-10-06T13:53:13.175 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a1081f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0a0c0b3a00 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:13.175 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a1081f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a0c13e4f0 0x7f0a0c0b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:13.175 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a1081f640 10 monclient(hunting): picked mon.noname-c con 0x7f0a0c13e4f0 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:13.175 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a1081f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0a0c0b1c40 0x7f0a0c14b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:13.175 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a1081f640 10 monclient(hunting): picked mon.noname-a con 0x7f0a0c0b1c40 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:13.175 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a1081f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0c0b3dd0 con 0x7f0a0c0b3a00 2023-10-06T13:53:13.175 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a1081f640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:13.175 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a1081f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0c0a93d0 con 0x7f0a0c0b1c40 2023-10-06T13:53:13.176 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a1081f640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:13.176 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a1081f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0c144ab0 con 0x7f0a0c13e4f0 2023-10-06T13:53:13.176 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a1081f640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:13.176 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a1081f640 10 monclient(hunting): authenticate will time out at 1321.357493s 2023-10-06T13:53:13.176 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.171+0000 7f0a03fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a0c13e4f0 0x7f0a0c0b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:13.176 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.172+0000 7f0a03fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a0c13e4f0 0x7f0a0c0b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:56606/0 (socket says 172.21.15.19:56606) 2023-10-06T13:53:13.176 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.172+0000 7f0a03fff640 1 -- 172.21.15.19:0/1576747688 learned_addr learned my addr 172.21.15.19:0/1576747688 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:13.176 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.172+0000 7f0a03fff640 10 monclient(hunting): get_auth_request con 0x7f0a0c13e4f0 auth_method 0 2023-10-06T13:53:13.176 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.172+0000 7f0a03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:13.176 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.172+0000 7f0a03fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:13.177 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.172+0000 7f0a03fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:13.177 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.172+0000 7f0a03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:13.177 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.172+0000 7f0a03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:13.177 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.172+0000 7f0a03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:13.178 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.174+0000 7f0a03fff640 10 monclient(hunting): handle_auth_done global_id 4264 payload 274 2023-10-06T13:53:13.178 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.174+0000 7f0a03fff640 1 -- 172.21.15.19:0/1576747688 >> v2:172.21.15.19:3300/0 conn(0x7f0a0c0b1c40 msgr2=0x7f0a0c14b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:13.178 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.174+0000 7f0a03fff640 1 --2- 172.21.15.19:0/1576747688 >> v2:172.21.15.19:3300/0 conn(0x7f0a0c0b1c40 0x7f0a0c14b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.178 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.174+0000 7f0a03fff640 1 -- 172.21.15.19:0/1576747688 >> v1:172.21.15.19:6789/0 conn(0x7f0a0c0b3a00 legacy=0x7f0a0c13dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:13.178 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.174+0000 7f0a03fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:13.178 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.174+0000 7f0a03fff640 1 monclient: found mon.noname-c 2023-10-06T13:53:13.178 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.174+0000 7f0a03fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:13.178 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.174+0000 7f0a03fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:53:13.178 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.174+0000 7f0a03fff640 1 -- 172.21.15.19:0/1576747688 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0a95d0 con 0x7f0a0c13e4f0 2023-10-06T13:53:13.178 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.176+0000 7f0a03fff640 1 --2- 172.21.15.19:0/1576747688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a0c13e4f0 0x7f0a0c0b1700 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0a8be0 tx=0x7f09f402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6dd7b523223a61d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:13.181 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a09d9b640 1 -- 172.21.15.19:0/1576747688 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4031870 con 0x7f0a0c13e4f0 2023-10-06T13:53:13.181 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a09d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:13.181 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a09d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:13.181 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a09d9b640 10 monclient: dump: 2023-10-06T13:53:13.182 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:53:13.182 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:13.182 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.182 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.182 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:13.182 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:13.182 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:13.182 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:13.182 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:13.182 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:53:13.182 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a09d9b640 10 monclient: _finish_auth 0 2023-10-06T13:53:13.183 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a09d9b640 10 monclient: _check_auth_tickets 2023-10-06T13:53:13.183 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a09d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:13.183 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a09d9b640 1 -- 172.21.15.19:0/1576747688 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f40319d0 con 0x7f0a0c13e4f0 2023-10-06T13:53:13.183 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a09d9b640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:13.183 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a09d9b640 1 -- 172.21.15.19:0/1576747688 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4031d60 con 0x7f0a0c13e4f0 2023-10-06T13:53:13.183 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a1081f640 5 monclient: authenticate success, global_id 4264 2023-10-06T13:53:13.183 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a1081f640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:13.183 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a1081f640 4 set_mon_vals no callback set 2023-10-06T13:53:13.183 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a09d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:13.183 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a09d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:13.183 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.179+0000 7f0a09d9b640 10 monclient: dump: 2023-10-06T13:53:13.183 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:53:13.184 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:13.184 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.184 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.184 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:13.184 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:13.184 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:13.184 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:13.184 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:13.184 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:53:13.187 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.180+0000 7f0a1081f640 10 monclient: shutdown 2023-10-06T13:53:13.187 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.180+0000 7f0a1081f640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:13.187 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.180+0000 7f0a1081f640 1 -- 172.21.15.19:0/1576747688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a0c13e4f0 msgr2=0x7f0a0c0b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:13.187 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.180+0000 7f0a1081f640 1 --2- 172.21.15.19:0/1576747688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a0c13e4f0 0x7f0a0c0b1700 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0a8be0 tx=0x7f09f402b910 comp rx=0 tx=0).stop 2023-10-06T13:53:13.187 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.180+0000 7f0a1081f640 1 -- 172.21.15.19:0/1576747688 shutdown_connections 2023-10-06T13:53:13.188 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.180+0000 7f0a1081f640 1 --2- 172.21.15.19:0/1576747688 >> v2:172.21.15.19:3300/0 conn(0x7f0a0c0b1c40 0x7f0a0c14b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.188 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.180+0000 7f0a1081f640 1 --2- 172.21.15.19:0/1576747688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a0c13e4f0 0x7f0a0c0b1700 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.188 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.180+0000 7f0a1081f640 1 -- 172.21.15.19:0/1576747688 >> 172.21.15.19:0/1576747688 conn(0x7f0a0c0659b0 msgr2=0x7f0a0c065db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:13.188 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.180+0000 7f0a1081f640 1 -- 172.21.15.19:0/1576747688 shutdown_connections 2023-10-06T13:53:13.188 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.180+0000 7f0a1081f640 1 -- 172.21.15.19:0/1576747688 wait complete. 2023-10-06T13:53:13.188 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.181+0000 7f0a1081f640 10 monclient: build_initial_monmap 2023-10-06T13:53:13.188 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.181+0000 7f0a1081f640 10 monclient: monmap: 2023-10-06T13:53:13.188 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-10-06T13:53:13.188 INFO:teuthology.misc.health.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:13.188 INFO:teuthology.misc.health.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:13.188 INFO:teuthology.misc.health.smithi019.stderr:created 0.000000 2023-10-06T13:53:13.188 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:13.189 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:13.189 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:13.189 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:13.189 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:13.189 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:53:13.189 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.181+0000 7f0a1081f640 1 librados: starting msgr at 2023-10-06T13:53:13.189 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.181+0000 7f0a1081f640 1 librados: starting objecter 2023-10-06T13:53:13.189 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.181+0000 7f0a1081f640 1 Processor -- start 2023-10-06T13:53:13.189 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.181+0000 7f0a1081f640 1 -- start start 2023-10-06T13:53:13.189 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.181+0000 7f0a1081f640 1 librados: setting wanted keys 2023-10-06T13:53:13.189 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.181+0000 7f0a1081f640 1 librados: calling monclient init 2023-10-06T13:53:13.189 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.181+0000 7f0a1081f640 10 monclient: init 2023-10-06T13:53:13.190 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:13.190 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T13:53:13.190 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a0c0b1c40 0x7f0a0c0add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:13.190 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 10 monclient(hunting): picked mon.noname-a con 0x7f0a0c0b1c40 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:13.190 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a0c0b3a00 0x7f0a0c0ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:13.190 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 10 monclient(hunting): picked mon.noname-b con 0x7f0a0c0b3a00 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:13.190 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0a0c13e4f0 0x7f0a0c1d9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:13.190 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 10 monclient(hunting): picked mon.noname-c con 0x7f0a0c13e4f0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:13.190 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:13.190 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1d90d0 con 0x7f0a0c0b1c40 2023-10-06T13:53:13.190 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:13.190 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0af700 con 0x7f0a0c0b3a00 2023-10-06T13:53:13.191 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:13.191 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0ac740 con 0x7f0a0c13e4f0 2023-10-06T13:53:13.191 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:13.191 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.182+0000 7f0a1081f640 10 monclient(hunting): authenticate will time out at 1321.368903s 2023-10-06T13:53:13.191 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a0a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a0c0b1c40 0x7f0a0c0add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:13.191 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a03fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a0c0b3a00 0x7f0a0c0ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:13.191 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a03fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a0c0b3a00 0x7f0a0c0ae240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:56618/0 (socket says 172.21.15.19:56618) 2023-10-06T13:53:13.191 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a03fff640 1 -- 172.21.15.19:0/1025281493 learned_addr learned my addr 172.21.15.19:0/1025281493 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:13.191 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a03fff640 10 monclient(hunting): get_auth_request con 0x7f0a0c0b3a00 auth_method 0 2023-10-06T13:53:13.191 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:13.191 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a03fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:13.191 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a03fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:13.192 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a0a59c640 10 monclient(hunting): get_auth_request con 0x7f0a0c0b1c40 auth_method 0 2023-10-06T13:53:13.192 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a0a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:13.192 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a0a59c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:13.192 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a0a59c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:13.192 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a0ad9d640 1 --2- 172.21.15.19:0/1025281493 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0a0c13e4f0 0x7f0a0c1d9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:13.192 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a0ad9d640 10 monclient(hunting): get_auth_request con 0x7f0a0c13e4f0 auth_method 0 2023-10-06T13:53:13.192 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a0ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:13.192 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a0ad9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:13.192 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a0ad9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:13.192 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:13.192 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:13.192 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:13.193 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a0a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:13.193 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.183+0000 7f0a0a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:13.193 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.184+0000 7f0a0a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:13.193 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.184+0000 7f0a0ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:13.193 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.184+0000 7f0a0ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:13.193 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.184+0000 7f0a0ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:13.194 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.186+0000 7f0a03fff640 10 monclient(hunting): handle_auth_done global_id 4267 payload 824 2023-10-06T13:53:13.195 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.186+0000 7f0a03fff640 1 -- 172.21.15.19:0/1025281493 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0a0c13e4f0 msgr2=0x7f0a0c1d9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:13.195 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.186+0000 7f0a03fff640 1 --2- 172.21.15.19:0/1025281493 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0a0c13e4f0 0x7f0a0c1d9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.195 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.186+0000 7f0a03fff640 1 -- 172.21.15.19:0/1025281493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a0c0b1c40 msgr2=0x7f0a0c0add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:13.195 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.186+0000 7f0a03fff640 1 --2- 172.21.15.19:0/1025281493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a0c0b1c40 0x7f0a0c0add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.195 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.186+0000 7f0a03fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:13.195 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.186+0000 7f0a03fff640 1 monclient: found mon.noname-b 2023-10-06T13:53:13.195 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.186+0000 7f0a03fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:13.195 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.186+0000 7f0a03fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:13.195 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.186+0000 7f0a03fff640 1 -- 172.21.15.19:0/1025281493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f40094d0 con 0x7f0a0c0b3a00 2023-10-06T13:53:13.195 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.188+0000 7f0a03fff640 1 --2- 172.21.15.19:0/1025281493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a0c0b3a00 0x7f0a0c0ae240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f09f4000c00 tx=0x7f09f40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:13.196 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.188+0000 7f0a02ffd640 10 client.?.objecter ms_handle_connect 0x7f0a0c0b3a00 2023-10-06T13:53:13.196 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.188+0000 7f0a02ffd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:13.196 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a02ffd640 1 -- 172.21.15.19:0/1025281493 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4031820 con 0x7f0a0c0b3a00 2023-10-06T13:53:13.196 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a02ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:13.196 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a02ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:13.196 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a02ffd640 10 monclient: dump: 2023-10-06T13:53:13.196 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:53:13.196 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:13.196 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.196 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.196 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:13.196 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:13.197 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:13.197 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:13.197 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:13.197 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:53:13.197 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a02ffd640 10 monclient: _finish_auth 0 2023-10-06T13:53:13.197 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a02ffd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:13.197 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a02ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:13.197 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a02ffd640 1 -- 172.21.15.19:0/1025281493 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f40443f0 con 0x7f0a0c0b3a00 2023-10-06T13:53:13.197 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a1081f640 5 monclient: authenticate success, global_id 4267 2023-10-06T13:53:13.197 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a1081f640 10 monclient: _renew_subs 2023-10-06T13:53:13.197 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a1081f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:13.197 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a1081f640 1 -- 172.21.15.19:0/1025281493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c0afe00 con 0x7f0a0c0b3a00 2023-10-06T13:53:13.197 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a02ffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:13.198 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a02ffd640 1 -- 172.21.15.19:0/1025281493 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f40447a0 con 0x7f0a0c0b3a00 2023-10-06T13:53:13.198 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a02ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:13.198 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a02ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:13.198 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a02ffd640 10 monclient: dump: 2023-10-06T13:53:13.198 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-10-06T13:53:13.198 INFO:teuthology.misc.health.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:13.198 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.198 INFO:teuthology.misc.health.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.198 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:13.198 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:13.198 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:13.198 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:13.199 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:13.199 INFO:teuthology.misc.health.smithi019.stderr: 2023-10-06T13:53:13.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a0959a640 4 set_mon_vals no callback set 2023-10-06T13:53:13.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a1081f640 10 client.4267.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:13.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a1081f640 10 monclient: _renew_subs 2023-10-06T13:53:13.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a1081f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:13.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a1081f640 1 -- 172.21.15.19:0/1025281493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c1d9c80 con 0x7f0a0c0b3a00 2023-10-06T13:53:13.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.191+0000 7f0a1081f640 1 librados: init done 2023-10-06T13:53:13.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.192+0000 7f0a1081f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:13.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.192+0000 7f0a1081f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:13.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.192+0000 7f0a1081f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:13.199 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.192+0000 7f0a1081f640 1 -- 172.21.15.19:0/1025281493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0c0ae780 con 0x7f0a0c0b3a00 2023-10-06T13:53:13.212 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.210+0000 7f0a02ffd640 1 -- 172.21.15.19:0/1025281493 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 88859+0+0 (secure 0 0 0) 0x7f09f404d450 con 0x7f0a0c0b3a00 2023-10-06T13:53:13.213 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.211+0000 7f0a02ffd640 1 --2- 172.21.15.19:0/1025281493 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f09ec06c180 0x7f09ec06e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:13.213 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.211+0000 7f0a0a59c640 1 --2- 172.21.15.19:0/1025281493 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f09ec06c180 0x7f09ec06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:13.213 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.211+0000 7f0a0a59c640 10 monclient: get_auth_request con 0x7f09ec06c180 auth_method 0 2023-10-06T13:53:13.213 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.211+0000 7f0a0a59c640 1 --2- 172.21.15.19:0/1025281493 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f09ec06c180 0x7f09ec06e630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0017f0 tx=0x7f09fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:13.213 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.211+0000 7f0a02ffd640 10 client.4267.objecter ms_handle_connect 0x7f09ec06c180 2023-10-06T13:53:13.214 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.213+0000 7f0a02ffd640 1 -- 172.21.15.19:0/1025281493 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f09f402d040 con 0x7f0a0c0b3a00 2023-10-06T13:53:13.215 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.213+0000 7f0a02ffd640 10 client.4267.objecter ms_dispatch 0x7f0a0c000b90 osd_map(19..19 src has 1..19) v4 2023-10-06T13:53:13.215 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.213+0000 7f0a02ffd640 3 client.4267.objecter handle_osd_map got epochs [19,19] > 0 2023-10-06T13:53:13.215 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.213+0000 7f0a02ffd640 3 client.4267.objecter handle_osd_map decoding full epoch 19 2023-10-06T13:53:13.215 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.213+0000 7f0a02ffd640 20 client.4267.objecter dump_active .. 0 homeless 2023-10-06T13:53:13.454 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.452+0000 7f0a02ffd640 1 -- 172.21.15.19:0/1025281493 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f09f4076bc0 con 0x7f0a0c0b3a00 2023-10-06T13:53:13.454 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.452+0000 7f0a02ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:13.454 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.452+0000 7f0a02ffd640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:13.639 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.637+0000 7f0a1081f640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:53:13.639 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.637+0000 7f0a1081f640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:53:13.639 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.637+0000 7f0a1081f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:13.639 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.637+0000 7f0a1081f640 1 -- 172.21.15.19:0/1025281493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a0c144ee0 con 0x7f0a0c0b3a00 2023-10-06T13:53:13.728 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.726+0000 7f0a02ffd640 1 -- 172.21.15.19:0/1025281493 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12334 (secure 0 0 0) 0x7f09f407a1e0 con 0x7f0a0c0b3a00 2023-10-06T13:53:13.728 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.726+0000 7f0a02ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:53:13.728 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.726+0000 7f0a02ffd640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:13.728 INFO:teuthology.misc.health.smithi019.stdout: 2023-10-06T13:53:13.729 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":19,"fsid":"c5a90f7c-81ec-43a3-ab47-af6ce5729ec5","created":"2023-10-06T13:52:19.516184+0000","modified":"2023-10-06T13:53:11.696628+0000","last_up_change":"2023-10-06T13:53:08.156870+0000","last_in_change":"2023-10-06T13:52:24.916896+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T13:53:07.784836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6cb75c5-86fb-48b9-895c-fb1d72bd4746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":86195},{"type":"v1","addr":"172.21.15.19:6818","nonce":86195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6821","nonce":86195},{"type":"v1","addr":"172.21.15.19:6823","nonce":86195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":86195},{"type":"v1","addr":"172.21.15.19:6831","nonce":86195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":86195},{"type":"v1","addr":"172.21.15.19:6827","nonce":86195}]},"public_addr":"172.21.15.19:6818/86195","cluster_addr":"172.21.15.19:6823/86195","heartbeat_back_addr":"172.21.15.19:6831/86195","heartbeat_front_addr":"172.21.15.19:6827/86195","state":["exists","up"]},{"osd":1,"uuid":"ba828cb4-a4f6-4fea-a02e-24d9cf597ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":86193},{"type":"v1","addr":"172.21.15.19:6819","nonce":86193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":86193},{"type":"v1","addr":"172.21.15.19:6822","nonce":86193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":86193},{"type":"v1","addr":"172.21.15.19:6830","nonce":86193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":86193},{"type":"v1","addr":"172.21.15.19:6826","nonce":86193}]},"public_addr":"172.21.15.19:6819/86193","cluster_addr":"172.21.15.19:6822/86193","heartbeat_back_addr":"172.21.15.19:6830/86193","heartbeat_front_addr":"172.21.15.19:6826/86193","state":["exists","up"]},{"osd":2,"uuid":"06bcfab5-d2f1-4447-ba44-143398707ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86194},{"type":"v1","addr":"172.21.15.19:6809","nonce":86194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":86194},{"type":"v1","addr":"172.21.15.19:6811","nonce":86194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":86194},{"type":"v1","addr":"172.21.15.19:6815","nonce":86194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":86194},{"type":"v1","addr":"172.21.15.19:6813","nonce":86194}]},"public_addr":"172.21.15.19:6809/86194","cluster_addr":"172.21.15.19:6811/86194","heartbeat_back_addr":"172.21.15.19:6815/86194","heartbeat_front_addr":"172.21.15.19:6813/86194","state":["exists","up"]},{"osd":3,"uuid":"7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":86196},{"type":"v1","addr":"172.21.15.19:6801","nonce":86196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86196},{"type":"v1","addr":"172.21.15.19:6803","nonce":86196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86196},{"type":"v1","addr":"172.21.15.19:6807","nonce":86196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86196},{"type":"v1","addr":"172.21.15.19:6805","nonce":86196}]},"public_addr":"172.21.15.19:6801/86196","cluster_addr":"172.21.15.19:6803/86196","heartbeat_back_addr":"172.21.15.19:6807/86196","heartbeat_front_addr":"172.21.15.19:6805/86196","state":["exists","up"]},{"osd":4,"uuid":"ad31d4df-e78c-4e32-bfd4-3fe6e70dd428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":86123},{"type":"v1","addr":"172.21.15.83:6827","nonce":86123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":86123},{"type":"v1","addr":"172.21.15.83:6829","nonce":86123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":86123},{"type":"v1","addr":"172.21.15.83:6833","nonce":86123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":86123},{"type":"v1","addr":"172.21.15.83:6831","nonce":86123}]},"public_addr":"172.21.15.83:6827/86123","cluster_addr":"172.21.15.83:6829/86123","heartbeat_back_addr":"172.21.15.83:6833/86123","heartbeat_front_addr":"172.21.15.83:6831/86123","state":["exists","up"]},{"osd":5,"uuid":"931b23f8-80c3-4595-8931-2568be479cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":86126},{"type":"v1","addr":"172.21.15.83:6819","nonce":86126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":86126},{"type":"v1","addr":"172.21.15.83:6821","nonce":86126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":86126},{"type":"v1","addr":"172.21.15.83:6825","nonce":86126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":86126},{"type":"v1","addr":"172.21.15.83:6823","nonce":86126}]},"public_addr":"172.21.15.83:6819/86126","cluster_addr":"172.21.15.83:6821/86126","heartbeat_back_addr":"172.21.15.83:6825/86126","heartbeat_front_addr":"172.21.15.83:6823/86126","state":["exists","up"]},{"osd":6,"uuid":"70dd2f95-93eb-448a-a65d-3a09bdea85a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":86124},{"type":"v1","addr":"172.21.15.83:6803","nonce":86124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":86124},{"type":"v1","addr":"172.21.15.83:6805","nonce":86124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":86124},{"type":"v1","addr":"172.21.15.83:6809","nonce":86124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":86124},{"type":"v1","addr":"172.21.15.83:6807","nonce":86124}]},"public_addr":"172.21.15.83:6803/86124","cluster_addr":"172.21.15.83:6805/86124","heartbeat_back_addr":"172.21.15.83:6809/86124","heartbeat_front_addr":"172.21.15.83:6807/86124","state":["exists","up"]},{"osd":7,"uuid":"9797eb84-1b2c-4280-ab13-bdb7b2007dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":86125},{"type":"v1","addr":"172.21.15.83:6811","nonce":86125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":86125},{"type":"v1","addr":"172.21.15.83:6813","nonce":86125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":86125},{"type":"v1","addr":"172.21.15.83:6817","nonce":86125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":86125},{"type":"v1","addr":"172.21.15.83:6815","nonce":86125}]},"public_addr":"172.21.15.83:6811/86125","cluster_addr":"172.21.15.83:6813/86125","heartbeat_back_addr":"172.21.15.83:6817/86125","heartbeat_front_addr":"172.21.15.83:6815/86125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:04.886714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:05.153337+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:04.794593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:04.049373+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:03.567476+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:02.921086+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:02.431856+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:02.863561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T13:53:13.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 10 librados: watch_flush enter 2023-10-06T13:53:13.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 10 librados: watch_flush exit 2023-10-06T13:53:13.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 20 client.4267.objecter shutdown clearing up homeless session... 2023-10-06T13:53:13.730 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 10 client.4267.objecter successfully canceled tick 2023-10-06T13:53:13.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 1 -- 172.21.15.19:0/1025281493 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f09ec06c180 msgr2=0x7f09ec06e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:13.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 1 --2- 172.21.15.19:0/1025281493 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f09ec06c180 0x7f09ec06e630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0017f0 tx=0x7f09fc009040 comp rx=0 tx=0).stop 2023-10-06T13:53:13.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 10 monclient: shutdown 2023-10-06T13:53:13.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:13.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 1 -- 172.21.15.19:0/1025281493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a0c0b3a00 msgr2=0x7f0a0c0ae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:13.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 1 --2- 172.21.15.19:0/1025281493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a0c0b3a00 0x7f0a0c0ae240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f09f4000c00 tx=0x7f09f40046c0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 1 -- 172.21.15.19:0/1025281493 shutdown_connections 2023-10-06T13:53:13.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 1 --2- 172.21.15.19:0/1025281493 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f09ec06c180 0x7f09ec06e630 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 1 --2- 172.21.15.19:0/1025281493 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0a0c13e4f0 0x7f0a0c1d9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 1 --2- 172.21.15.19:0/1025281493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a0c0b3a00 0x7f0a0c0ae240 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 1 --2- 172.21.15.19:0/1025281493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a0c0b1c40 0x7f0a0c0add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.731 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 1 -- 172.21.15.19:0/1025281493 >> 172.21.15.19:0/1025281493 conn(0x7f0a0c0659b0 msgr2=0x7f0a0c0b70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:13.732 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 1 -- 172.21.15.19:0/1025281493 shutdown_connections 2023-10-06T13:53:13.732 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 1 -- 172.21.15.19:0/1025281493 wait complete. 2023-10-06T13:53:13.732 INFO:teuthology.misc.health.smithi019.stderr:2023-10-06T13:53:13.728+0000 7f0a1081f640 1 librados: shutdown 2023-10-06T13:53:13.739 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T13:53:13.740 INFO:tasks.ceph:Creating RBD pool 2023-10-06T13:53:13.740 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T13:53:13.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.867+0000 7f3c1d3c4640 10 monclient: get_monmap_and_config 2023-10-06T13:53:13.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.867+0000 7f3c1d3c4640 10 monclient: build_initial_monmap 2023-10-06T13:53:13.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.867+0000 7f3c1d3c4640 10 monclient: monmap: 2023-10-06T13:53:13.871 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:13.872 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:13.872 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:13.868642+0000 2023-10-06T13:53:13.872 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:13.868642+0000 2023-10-06T13:53:13.872 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:13.872 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:13.872 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:13.872 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:13.872 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:13.872 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:13.872 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:13.872 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:13.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 1 Processor -- start 2023-10-06T13:53:13.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 1 -- start start 2023-10-06T13:53:13.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 10 monclient: init 2023-10-06T13:53:13.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:13.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2023-10-06T13:53:13.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3c1806b6d0 0x7f3c18144200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:13.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 10 monclient(hunting): picked mon.noname-a con 0x7f3c1806b6d0 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:13.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3c18144740 0x7f3c1813df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:13.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 10 monclient(hunting): picked mon.noname-b con 0x7f3c18144740 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:13.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3c1813e460 0x7f3c180b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:13.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 10 monclient(hunting): picked mon.noname-c con 0x7f3c1813e460 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:13.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:13.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c1806baa0 con 0x7f3c1806b6d0 2023-10-06T13:53:13.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:13.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c180b2530 con 0x7f3c1813e460 2023-10-06T13:53:13.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:13.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180b2810 con 0x7f3c18144740 2023-10-06T13:53:13.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:13.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.868+0000 7f3c1d3c4640 10 monclient(hunting): authenticate will time out at 1322.055257s 2023-10-06T13:53:13.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.869+0000 7f3c177fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3c1813e460 0x7f3c180b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:13.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.869+0000 7f3c177fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3c1813e460 0x7f3c180b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:56630/0 (socket says 172.21.15.19:56630) 2023-10-06T13:53:13.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.869+0000 7f3c177fe640 1 -- 172.21.15.19:0/662932971 learned_addr learned my addr 172.21.15.19:0/662932971 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:13.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.869+0000 7f3c177fe640 10 monclient(hunting): get_auth_request con 0x7f3c1813e460 auth_method 0 2023-10-06T13:53:13.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.869+0000 7f3c177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:13.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.869+0000 7f3c177fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:13.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.869+0000 7f3c177fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:13.875 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.870+0000 7f3c177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:13.875 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.870+0000 7f3c177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:13.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.870+0000 7f3c177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:13.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.871+0000 7f3c177fe640 10 monclient(hunting): handle_auth_done global_id 4276 payload 274 2023-10-06T13:53:13.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.871+0000 7f3c177fe640 1 -- 172.21.15.19:0/662932971 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3c18144740 msgr2=0x7f3c1813df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:13.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.871+0000 7f3c177fe640 1 --2- 172.21.15.19:0/662932971 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3c18144740 0x7f3c1813df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.871+0000 7f3c177fe640 1 -- 172.21.15.19:0/662932971 >> v2:172.21.15.19:3300/0 conn(0x7f3c1806b6d0 msgr2=0x7f3c18144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:13.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.871+0000 7f3c177fe640 1 --2- 172.21.15.19:0/662932971 >> v2:172.21.15.19:3300/0 conn(0x7f3c1806b6d0 0x7f3c18144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.871+0000 7f3c177fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:13.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.871+0000 7f3c177fe640 1 monclient: found mon.noname-c 2023-10-06T13:53:13.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.871+0000 7f3c177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:13.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.871+0000 7f3c177fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:53:13.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.871+0000 7f3c177fe640 1 -- 172.21.15.19:0/662932971 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c180aeff0 con 0x7f3c1813e460 2023-10-06T13:53:13.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.874+0000 7f3c177fe640 1 --2- 172.21.15.19:0/662932971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1813e460 0x7f3c180b3740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c180add90 tx=0x7f3c1002bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3105dd01abca5c84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:13.879 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c15ffb640 1 -- 172.21.15.19:0/662932971 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1002ca70 con 0x7f3c1813e460 2023-10-06T13:53:13.879 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c15ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:13.879 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c15ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:13.879 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c15ffb640 10 monclient: dump: 2023-10-06T13:53:13.879 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:13.879 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:13.879 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.879 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.880 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:13.880 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:13.880 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:13.880 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:13.880 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:13.880 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:13.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c15ffb640 10 monclient: _finish_auth 0 2023-10-06T13:53:13.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c15ffb640 10 monclient: _check_auth_tickets 2023-10-06T13:53:13.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c15ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:13.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c15ffb640 1 -- 172.21.15.19:0/662932971 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1002cbd0 con 0x7f3c1813e460 2023-10-06T13:53:13.881 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c1d3c4640 5 monclient: authenticate success, global_id 4276 2023-10-06T13:53:13.881 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c1d3c4640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:13.881 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c15ffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:13.881 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c1d3c4640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:13.881 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c15ffb640 1 -- 172.21.15.19:0/662932971 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c100343f0 con 0x7f3c1813e460 2023-10-06T13:53:13.881 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c1d3c4640 4 set_mon_vals no callback set 2023-10-06T13:53:13.881 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c15ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:13.881 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c15ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:13.881 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.877+0000 7f3c15ffb640 10 monclient: dump: 2023-10-06T13:53:13.881 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:13.881 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:13.881 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.881 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.882 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:13.882 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:13.882 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:13.882 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:13.882 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:13.882 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:13.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.878+0000 7f3c1d3c4640 10 monclient: shutdown 2023-10-06T13:53:13.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.878+0000 7f3c1d3c4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:13.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.878+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/662932971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1813e460 msgr2=0x7f3c180b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:13.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.878+0000 7f3c1d3c4640 1 --2- 172.21.15.19:0/662932971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1813e460 0x7f3c180b3740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c180add90 tx=0x7f3c1002bed0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.878+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/662932971 shutdown_connections 2023-10-06T13:53:13.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.878+0000 7f3c1d3c4640 1 --2- 172.21.15.19:0/662932971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1813e460 0x7f3c180b3740 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.878+0000 7f3c1d3c4640 1 --2- 172.21.15.19:0/662932971 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3c18144740 0x7f3c1813df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.878+0000 7f3c1d3c4640 1 --2- 172.21.15.19:0/662932971 >> v2:172.21.15.19:3300/0 conn(0x7f3c1806b6d0 0x7f3c18144200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.878+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/662932971 >> 172.21.15.19:0/662932971 conn(0x7f3c180665b0 msgr2=0x7f3c1806ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:13.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.878+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/662932971 shutdown_connections 2023-10-06T13:53:13.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.878+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/662932971 wait complete. 2023-10-06T13:53:13.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.879+0000 7f3c1d3c4640 10 monclient: build_initial_monmap 2023-10-06T13:53:13.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.879+0000 7f3c1d3c4640 10 monclient: monmap: 2023-10-06T13:53:13.887 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:13.887 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:13.887 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:13.888 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:13.888 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:13.888 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:13.888 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:13.888 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:13.888 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:13.888 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:13.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.879+0000 7f3c1d3c4640 1 librados: starting msgr at 2023-10-06T13:53:13.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.879+0000 7f3c1d3c4640 1 librados: starting objecter 2023-10-06T13:53:13.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.880+0000 7f3c1d3c4640 1 Processor -- start 2023-10-06T13:53:13.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.880+0000 7f3c1d3c4640 1 -- start start 2023-10-06T13:53:13.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.880+0000 7f3c1d3c4640 1 librados: setting wanted keys 2023-10-06T13:53:13.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.880+0000 7f3c1d3c4640 1 librados: calling monclient init 2023-10-06T13:53:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.880+0000 7f3c1d3c4640 10 monclient: init 2023-10-06T13:53:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T13:53:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1806b6d0 0x7f3c1813b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 10 monclient(hunting): picked mon.noname-b con 0x7f3c1806b6d0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3c1813e460 0x7f3c180ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 10 monclient(hunting): picked mon.noname-c con 0x7f3c1813e460 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c18144740 0x7f3c181413e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:13.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 10 monclient(hunting): picked mon.noname-a con 0x7f3c18144740 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:13.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:13.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180a9880 con 0x7f3c18144740 2023-10-06T13:53:13.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:13.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180aab70 con 0x7f3c1806b6d0 2023-10-06T13:53:13.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c18141920 con 0x7f3c1813e460 2023-10-06T13:53:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c1d3c4640 10 monclient(hunting): authenticate will time out at 1322.067557s 2023-10-06T13:53:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.881+0000 7f3c16ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1806b6d0 0x7f3c1813b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.882+0000 7f3c16ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1806b6d0 0x7f3c1813b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:56638/0 (socket says 172.21.15.19:56638) 2023-10-06T13:53:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.882+0000 7f3c16ffd640 1 -- 172.21.15.19:0/742423155 learned_addr learned my addr 172.21.15.19:0/742423155 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.882+0000 7f3c16ffd640 10 monclient(hunting): get_auth_request con 0x7f3c1806b6d0 auth_method 0 2023-10-06T13:53:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.882+0000 7f3c16ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.882+0000 7f3c16ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.882+0000 7f3c16ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.882+0000 7f3c16ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:13.894 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.882+0000 7f3c16ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:13.894 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.882+0000 7f3c16ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:13.894 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.884+0000 7f3c16ffd640 10 monclient(hunting): handle_auth_done global_id 4279 payload 824 2023-10-06T13:53:13.894 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.885+0000 7f3c16ffd640 1 -- 172.21.15.19:0/742423155 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3c1813e460 msgr2=0x7f3c180ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:13.894 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.885+0000 7f3c16ffd640 1 --2- 172.21.15.19:0/742423155 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3c1813e460 0x7f3c180ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.895 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.885+0000 7f3c16ffd640 1 -- 172.21.15.19:0/742423155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c18144740 msgr2=0x7f3c181413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:13.895 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.885+0000 7f3c16ffd640 1 --2- 172.21.15.19:0/742423155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c18144740 0x7f3c181413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:13.895 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.885+0000 7f3c16ffd640 10 monclient: _finish_hunting 0 2023-10-06T13:53:13.895 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.885+0000 7f3c16ffd640 1 monclient: found mon.noname-b 2023-10-06T13:53:13.895 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.885+0000 7f3c16ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:13.895 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.885+0000 7f3c16ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:13.895 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.885+0000 7f3c16ffd640 1 -- 172.21.15.19:0/742423155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c10009b30 con 0x7f3c1806b6d0 2023-10-06T13:53:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.896+0000 7f3c16ffd640 1 --2- 172.21.15.19:0/742423155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1806b6d0 0x7f3c1813b650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800b4a0 tx=0x7f3c0800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.896+0000 7f3bff7fe640 10 client.?.objecter ms_handle_connect 0x7f3c1806b6d0 2023-10-06T13:53:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.896+0000 7f3bff7fe640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:13.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.900+0000 7f3bff7fe640 1 -- 172.21.15.19:0/742423155 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08010610 con 0x7f3c1806b6d0 2023-10-06T13:53:13.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.900+0000 7f3bff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:13.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.900+0000 7f3bff7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:13.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.900+0000 7f3bff7fe640 10 monclient: dump: 2023-10-06T13:53:13.903 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:13.903 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:13.904 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3bff7fe640 10 monclient: _finish_auth 0 2023-10-06T13:53:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3bff7fe640 10 monclient: _check_auth_tickets 2023-10-06T13:53:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3bff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3bff7fe640 1 -- 172.21.15.19:0/742423155 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c08010770 con 0x7f3c1806b6d0 2023-10-06T13:53:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3c1d3c4640 5 monclient: authenticate success, global_id 4279 2023-10-06T13:53:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3c1d3c4640 10 monclient: _renew_subs 2023-10-06T13:53:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3c1d3c4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/742423155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c181d89d0 con 0x7f3c1806b6d0 2023-10-06T13:53:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3bff7fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:13.906 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3bff7fe640 1 -- 172.21.15.19:0/742423155 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08010b20 con 0x7f3c1806b6d0 2023-10-06T13:53:13.906 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3bff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:13.906 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3bff7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:13.906 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3bff7fe640 10 monclient: dump: 2023-10-06T13:53:13.906 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:13.908 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:13.908 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:13.908 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:13.908 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3c14ff9640 4 set_mon_vals no callback set 2023-10-06T13:53:13.908 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3c1d3c4640 10 client.4279.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3c1d3c4640 10 monclient: _renew_subs 2023-10-06T13:53:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3c1d3c4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/742423155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1813ad10 con 0x7f3c1806b6d0 2023-10-06T13:53:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.901+0000 7f3c1d3c4640 1 librados: init done 2023-10-06T13:53:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.902+0000 7f3c1d3c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.902+0000 7f3c1d3c4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.902+0000 7f3c1d3c4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:13.911 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:13.902+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/742423155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1804e000 con 0x7f3c1806b6d0 2023-10-06T13:53:14.147 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.145+0000 7f3bff7fe640 1 -- 172.21.15.19:0/742423155 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 88859+0+0 (secure 0 0 0) 0x7f3c08010c80 con 0x7f3c1806b6d0 2023-10-06T13:53:14.147 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.145+0000 7f3bff7fe640 1 --2- 172.21.15.19:0/742423155 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3c0406c180 0x7f3c0406e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:14.148 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.145+0000 7f3c167fc640 1 --2- 172.21.15.19:0/742423155 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3c0406c180 0x7f3c0406e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:14.148 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.145+0000 7f3c167fc640 10 monclient: get_auth_request con 0x7f3c0406c180 auth_method 0 2023-10-06T13:53:14.148 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.146+0000 7f3c167fc640 1 --2- 172.21.15.19:0/742423155 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3c0406c180 0x7f3c0406e630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3c18141d90 tx=0x7f3c0c006bf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:14.148 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.146+0000 7f3bff7fe640 10 client.4279.objecter ms_handle_connect 0x7f3c0406c180 2023-10-06T13:53:14.149 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.148+0000 7f3bff7fe640 1 -- 172.21.15.19:0/742423155 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3c08058630 con 0x7f3c1806b6d0 2023-10-06T13:53:14.150 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.148+0000 7f3bff7fe640 10 client.4279.objecter ms_dispatch 0x7f3c18000b90 osd_map(19..19 src has 1..19) v4 2023-10-06T13:53:14.150 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.148+0000 7f3bff7fe640 3 client.4279.objecter handle_osd_map got epochs [19,19] > 0 2023-10-06T13:53:14.150 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.148+0000 7f3bff7fe640 3 client.4279.objecter handle_osd_map decoding full epoch 19 2023-10-06T13:53:14.150 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.148+0000 7f3bff7fe640 20 client.4279.objecter dump_active .. 0 homeless 2023-10-06T13:53:14.160 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.158+0000 7f3bff7fe640 1 -- 172.21.15.19:0/742423155 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f3c08054270 con 0x7f3c1806b6d0 2023-10-06T13:53:14.160 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.158+0000 7f3bff7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:14.160 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.158+0000 7f3bff7fe640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.341+0000 7f3c1d3c4640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2023-10-06T13:53:14.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.341+0000 7f3c1d3c4640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2023-10-06T13:53:14.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.341+0000 7f3c1d3c4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:14.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:14.341+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/742423155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f3c18144bb0 con 0x7f3c1806b6d0 2023-10-06T13:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.027+0000 7f3bff7fe640 1 -- 172.21.15.19:0/742423155 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v20) v1 ==== 109+0+0 (secure 0 0 0) 0x7f3c0801e020 con 0x7f3c1806b6d0 2023-10-06T13:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.027+0000 7f3bff7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2023-10-06T13:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.027+0000 7f3bff7fe640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2023-10-06T13:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'rbd' created 2023-10-06T13:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.029+0000 7f3c1d3c4640 10 librados: watch_flush enter 2023-10-06T13:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.029+0000 7f3c1d3c4640 10 librados: watch_flush exit 2023-10-06T13:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.029+0000 7f3c1d3c4640 20 client.4279.objecter shutdown clearing up homeless session... 2023-10-06T13:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.029+0000 7f3c1d3c4640 10 client.4279.objecter successfully canceled tick 2023-10-06T13:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.029+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/742423155 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3c0406c180 msgr2=0x7f3c0406e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.029+0000 7f3c1d3c4640 1 --2- 172.21.15.19:0/742423155 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3c0406c180 0x7f3c0406e630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3c18141d90 tx=0x7f3c0c006bf0 comp rx=0 tx=0).stop 2023-10-06T13:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.029+0000 7f3c1d3c4640 10 monclient: shutdown 2023-10-06T13:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.029+0000 7f3c1d3c4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.029+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/742423155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1806b6d0 msgr2=0x7f3c1813b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.029+0000 7f3c1d3c4640 1 --2- 172.21.15.19:0/742423155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1806b6d0 0x7f3c1813b650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800b4a0 tx=0x7f3c0800b960 comp rx=0 tx=0).stop 2023-10-06T13:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.029+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/742423155 shutdown_connections 2023-10-06T13:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.029+0000 7f3c1d3c4640 1 --2- 172.21.15.19:0/742423155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c18144740 0x7f3c181413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.030+0000 7f3c1d3c4640 1 --2- 172.21.15.19:0/742423155 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3c0406c180 0x7f3c0406e630 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.030+0000 7f3c1d3c4640 1 --2- 172.21.15.19:0/742423155 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3c1813e460 0x7f3c180ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.030+0000 7f3c1d3c4640 1 --2- 172.21.15.19:0/742423155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1806b6d0 0x7f3c1813b650 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.030+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/742423155 >> 172.21.15.19:0/742423155 conn(0x7f3c180665b0 msgr2=0x7f3c180af780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.030+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/742423155 shutdown_connections 2023-10-06T13:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.030+0000 7f3c1d3c4640 1 -- 172.21.15.19:0/742423155 wait complete. 2023-10-06T13:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.030+0000 7f3c1d3c4640 1 librados: shutdown 2023-10-06T13:53:15.046 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T13:53:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.158+0000 7f6999bd0640 10 monclient: get_monmap_and_config 2023-10-06T13:53:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.158+0000 7f6999bd0640 10 monclient: build_initial_monmap 2023-10-06T13:53:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.158+0000 7f6999bd0640 10 monclient: monmap: 2023-10-06T13:53:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:15.159347+0000 2023-10-06T13:53:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:15.159347+0000 2023-10-06T13:53:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:15.162 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 1 Processor -- start 2023-10-06T13:53:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 1 -- start start 2023-10-06T13:53:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 10 monclient: init 2023-10-06T13:53:15.164 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:15.164 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 10 monclient: _add_conns ranks=[0,3,4,2,1] 2023-10-06T13:53:15.164 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f69940b3990 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:15.164 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f69940b3d60 0x7f69940af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:15.165 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 10 monclient(hunting): picked mon.noname-a con 0x7f69940b3d60 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:15.165 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69940b00c0 0x7f69940b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:15.165 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 10 monclient(hunting): picked mon.noname-c con 0x7f69940b00c0 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:15.165 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69940aebe0 con 0x7f69940b3990 2023-10-06T13:53:15.165 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:15.165 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69940b0a60 con 0x7f69940b3d60 2023-10-06T13:53:15.165 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:15.165 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6994140dd0 con 0x7f69940b00c0 2023-10-06T13:53:15.165 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:15.165 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.159+0000 7f6999bd0640 10 monclient(hunting): authenticate will time out at 1323.346077s 2023-10-06T13:53:15.165 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.160+0000 7f6993fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69940b00c0 0x7f69940b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.160+0000 7f6992ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f69940b3d60 0x7f69940af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.160+0000 7f69937fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f69940b3990 0x7f6994141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57838/0 (socket says 172.21.15.19:57838) 2023-10-06T13:53:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.160+0000 7f69937fe640 1 -- 172.21.15.19:0/2667445944 learned_addr learned my addr 172.21.15.19:0/2667445944 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.160+0000 7f6993fff640 10 monclient(hunting): get_auth_request con 0x7f69940b00c0 auth_method 0 2023-10-06T13:53:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.160+0000 7f6993fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.160+0000 7f6993fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.160+0000 7f6993fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.161+0000 7f6992ffd640 10 monclient(hunting): get_auth_request con 0x7f69940b3d60 auth_method 0 2023-10-06T13:53:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.161+0000 7f6992ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.161+0000 7f6992ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.161+0000 7f6992ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.161+0000 7f6993fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:15.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.161+0000 7f6993fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:15.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.161+0000 7f6993fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:15.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.162+0000 7f6993fff640 10 monclient(hunting): handle_auth_done global_id 4282 payload 274 2023-10-06T13:53:15.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.162+0000 7f6993fff640 1 -- 172.21.15.19:0/2667445944 >> v2:172.21.15.19:3300/0 conn(0x7f69940b3d60 msgr2=0x7f69940af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:15.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.162+0000 7f6993fff640 1 --2- 172.21.15.19:0/2667445944 >> v2:172.21.15.19:3300/0 conn(0x7f69940b3d60 0x7f69940af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:15.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.162+0000 7f6993fff640 1 -- 172.21.15.19:0/2667445944 >> v1:172.21.15.19:6789/0 conn(0x7f69940b3990 legacy=0x7f6994141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:15.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.162+0000 7f6993fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:15.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.162+0000 7f6993fff640 1 monclient: found mon.noname-c 2023-10-06T13:53:15.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.162+0000 7f6993fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:15.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.162+0000 7f6993fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:53:15.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.162+0000 7f6993fff640 1 -- 172.21.15.19:0/2667445944 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6994141070 con 0x7f69940b00c0 2023-10-06T13:53:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.164+0000 7f6993fff640 1 --2- 172.21.15.19:0/2667445944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b00c0 0x7f69940b0520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f69940ae820 tx=0x7f698c02be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=632f1d2347890319 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f69927fc640 1 -- 172.21.15.19:0/2667445944 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c02cbf0 con 0x7f69940b00c0 2023-10-06T13:53:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f69927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f69927fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f69927fc640 10 monclient: dump: 2023-10-06T13:53:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:15.172 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:15.172 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:15.172 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:15.172 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:15.172 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:15.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f69927fc640 10 monclient: _finish_auth 0 2023-10-06T13:53:15.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f69927fc640 10 monclient: _check_auth_tickets 2023-10-06T13:53:15.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f69927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:15.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f69927fc640 1 -- 172.21.15.19:0/2667445944 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698c02cd50 con 0x7f69940b00c0 2023-10-06T13:53:15.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f69927fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:15.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f69927fc640 1 -- 172.21.15.19:0/2667445944 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c034660 con 0x7f69940b00c0 2023-10-06T13:53:15.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f6999bd0640 5 monclient: authenticate success, global_id 4282 2023-10-06T13:53:15.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f6999bd0640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:15.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f6999bd0640 4 set_mon_vals no callback set 2023-10-06T13:53:15.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f69927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:15.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f69927fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:15.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.168+0000 7f69927fc640 10 monclient: dump: 2023-10-06T13:53:15.173 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:15.173 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:15.173 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:15.174 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:15.174 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:15.174 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:15.174 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:15.174 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:15.174 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:15.174 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.169+0000 7f6999bd0640 10 monclient: shutdown 2023-10-06T13:53:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.169+0000 7f6999bd0640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.169+0000 7f6999bd0640 1 -- 172.21.15.19:0/2667445944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b00c0 msgr2=0x7f69940b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.169+0000 7f6999bd0640 1 --2- 172.21.15.19:0/2667445944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b00c0 0x7f69940b0520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f69940ae820 tx=0x7f698c02be40 comp rx=0 tx=0).stop 2023-10-06T13:53:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.169+0000 7f6999bd0640 1 -- 172.21.15.19:0/2667445944 shutdown_connections 2023-10-06T13:53:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.169+0000 7f6999bd0640 1 --2- 172.21.15.19:0/2667445944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b00c0 0x7f69940b0520 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.169+0000 7f6999bd0640 1 --2- 172.21.15.19:0/2667445944 >> v2:172.21.15.19:3300/0 conn(0x7f69940b3d60 0x7f69940af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.169+0000 7f6999bd0640 1 -- 172.21.15.19:0/2667445944 >> 172.21.15.19:0/2667445944 conn(0x7f69940b6e00 msgr2=0x7f69940b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.169+0000 7f6999bd0640 1 -- 172.21.15.19:0/2667445944 shutdown_connections 2023-10-06T13:53:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.169+0000 7f6999bd0640 1 -- 172.21.15.19:0/2667445944 wait complete. 2023-10-06T13:53:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.170+0000 7f6999bd0640 10 monclient: build_initial_monmap 2023-10-06T13:53:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.170+0000 7f6999bd0640 10 monclient: monmap: 2023-10-06T13:53:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:15.179 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.170+0000 7f6999bd0640 1 librados: starting msgr at 2023-10-06T13:53:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.170+0000 7f6999bd0640 1 librados: starting objecter 2023-10-06T13:53:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.170+0000 7f6999bd0640 1 Processor -- start 2023-10-06T13:53:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.170+0000 7f6999bd0640 1 -- start start 2023-10-06T13:53:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.170+0000 7f6999bd0640 1 librados: setting wanted keys 2023-10-06T13:53:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.170+0000 7f6999bd0640 1 librados: calling monclient init 2023-10-06T13:53:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.170+0000 7f6999bd0640 10 monclient: init 2023-10-06T13:53:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 10 monclient: _add_conns ranks=[2,1,0] 2023-10-06T13:53:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f69940b00c0 0x7f69940aa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 10 monclient(hunting): picked mon.noname-c con 0x7f69940b00c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b3990 0x7f6994143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 10 monclient(hunting): picked mon.noname-b con 0x7f69940b3990 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69940b3d60 0x7f699413a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 10 monclient(hunting): picked mon.noname-a con 0x7f69940b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69940aae20 con 0x7f69940b3d60 2023-10-06T13:53:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6994144b80 con 0x7f69940b3990 2023-10-06T13:53:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69940aa900 con 0x7f69940b00c0 2023-10-06T13:53:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6999bd0640 10 monclient(hunting): authenticate will time out at 1323.357902s 2023-10-06T13:53:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.171+0000 7f6992ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b3990 0x7f6994143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.172+0000 7f69937fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f69940b00c0 0x7f69940aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.172+0000 7f6992ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b3990 0x7f6994143de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39194/0 (socket says 172.21.15.19:39194) 2023-10-06T13:53:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.172+0000 7f6992ffd640 1 -- 172.21.15.19:0/3623102548 learned_addr learned my addr 172.21.15.19:0/3623102548 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.172+0000 7f6992ffd640 10 monclient(hunting): get_auth_request con 0x7f69940b3990 auth_method 0 2023-10-06T13:53:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.172+0000 7f6992ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.172+0000 7f6992ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.172+0000 7f6992ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.172+0000 7f69937fe640 10 monclient(hunting): get_auth_request con 0x7f69940b00c0 auth_method 0 2023-10-06T13:53:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.172+0000 7f69937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.172+0000 7f69937fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.172+0000 7f69937fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.172+0000 7f6992ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.172+0000 7f6992ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.172+0000 7f6992ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.173+0000 7f69937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.173+0000 7f69937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.173+0000 7f69937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.175+0000 7f6992ffd640 10 monclient(hunting): handle_auth_done global_id 4285 payload 824 2023-10-06T13:53:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.175+0000 7f6992ffd640 1 -- 172.21.15.19:0/3623102548 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f69940b00c0 msgr2=0x7f69940aa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.175+0000 7f6992ffd640 1 --2- 172.21.15.19:0/3623102548 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f69940b00c0 0x7f69940aa320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.175+0000 7f6992ffd640 1 -- 172.21.15.19:0/3623102548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69940b3d60 msgr2=0x7f699413a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.175+0000 7f6992ffd640 1 --2- 172.21.15.19:0/3623102548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69940b3d60 0x7f699413a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.175+0000 7f6992ffd640 10 monclient: _finish_hunting 0 2023-10-06T13:53:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.175+0000 7f6992ffd640 1 monclient: found mon.noname-b 2023-10-06T13:53:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.175+0000 7f6992ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.175+0000 7f6992ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.175+0000 7f6992ffd640 1 -- 172.21.15.19:0/3623102548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c009a70 con 0x7f69940b3990 2023-10-06T13:53:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.177+0000 7f6992ffd640 1 --2- 172.21.15.19:0/3623102548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b3990 0x7f6994143de0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f698800d5d0 tx=0x7f698800da90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.177+0000 7f697bfff640 10 client.?.objecter ms_handle_connect 0x7f69940b3990 2023-10-06T13:53:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.177+0000 7f697bfff640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f697bfff640 1 -- 172.21.15.19:0/3623102548 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988004240 con 0x7f69940b3990 2023-10-06T13:53:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f697bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f697bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f697bfff640 10 monclient: dump: 2023-10-06T13:53:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:15.186 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f697bfff640 10 monclient: _finish_auth 0 2023-10-06T13:53:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f697bfff640 10 monclient: _check_auth_tickets 2023-10-06T13:53:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f697bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f697bfff640 1 -- 172.21.15.19:0/3623102548 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69880043a0 con 0x7f69940b3990 2023-10-06T13:53:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f6999bd0640 5 monclient: authenticate success, global_id 4285 2023-10-06T13:53:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f6999bd0640 10 monclient: _renew_subs 2023-10-06T13:53:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f6999bd0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f6999bd0640 1 -- 172.21.15.19:0/3623102548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69940a9820 con 0x7f69940b3990 2023-10-06T13:53:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f697bfff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f697bfff640 1 -- 172.21.15.19:0/3623102548 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988003830 con 0x7f69940b3990 2023-10-06T13:53:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f697bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f697bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f697bfff640 10 monclient: dump: 2023-10-06T13:53:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:15.188 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f6991ffb640 4 set_mon_vals no callback set 2023-10-06T13:53:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f6999bd0640 10 client.4285.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f6999bd0640 10 monclient: _renew_subs 2023-10-06T13:53:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f6999bd0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f6999bd0640 1 -- 172.21.15.19:0/3623102548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699413a900 con 0x7f69940b3990 2023-10-06T13:53:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.182+0000 7f6999bd0640 1 librados: init done 2023-10-06T13:53:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.183+0000 7f6999bd0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.183+0000 7f6999bd0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.183+0000 7f6999bd0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.183+0000 7f6999bd0640 1 -- 172.21.15.19:0/3623102548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69940678e0 con 0x7f69940b3990 2023-10-06T13:53:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.206+0000 7f697bfff640 1 -- 172.21.15.19:0/3623102548 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 88859+0+0 (secure 0 0 0) 0x7f6988003990 con 0x7f69940b3990 2023-10-06T13:53:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.206+0000 7f697bfff640 1 --2- 172.21.15.19:0/3623102548 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f698006c180 0x7f698006e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.206+0000 7f69937fe640 1 --2- 172.21.15.19:0/3623102548 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f698006c180 0x7f698006e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.206+0000 7f69937fe640 10 monclient: get_auth_request con 0x7f698006c180 auth_method 0 2023-10-06T13:53:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.207+0000 7f69937fe640 1 --2- 172.21.15.19:0/3623102548 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f698006c180 0x7f698006e630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6984007fd0 tx=0x7f6984007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.207+0000 7f697bfff640 10 client.4285.objecter ms_handle_connect 0x7f698006c180 2023-10-06T13:53:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.209+0000 7f697bfff640 1 -- 172.21.15.19:0/3623102548 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f69880591a0 con 0x7f69940b3990 2023-10-06T13:53:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.209+0000 7f697bfff640 10 client.4285.objecter ms_dispatch 0x7f6994000b90 osd_map(20..20 src has 1..20) v4 2023-10-06T13:53:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.209+0000 7f697bfff640 3 client.4285.objecter handle_osd_map got epochs [20,20] > 0 2023-10-06T13:53:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.209+0000 7f697bfff640 3 client.4285.objecter handle_osd_map decoding full epoch 20 2023-10-06T13:53:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.209+0000 7f697bfff640 20 client.4285.objecter dump_active .. 0 homeless 2023-10-06T13:53:15.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.451+0000 7f697bfff640 1 -- 172.21.15.19:0/3623102548 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f6988054de0 con 0x7f69940b3990 2023-10-06T13:53:15.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.451+0000 7f697bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:15.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.451+0000 7f697bfff640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:15.642 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.640+0000 7f6999bd0640 10 start_mon_command cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2023-10-06T13:53:15.643 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.640+0000 7f6999bd0640 10 monclient: _send_command 2 [{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2023-10-06T13:53:15.643 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.641+0000 7f6999bd0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:15.643 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:15.641+0000 7f6999bd0640 1 -- 172.21.15.19:0/3623102548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f69940aaac0 con 0x7f69940b3990 2023-10-06T13:53:16.447 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.445+0000 7f697bfff640 1 -- 172.21.15.19:0/3623102548 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f69880551b0 con 0x7f69940b3990 2023-10-06T13:53:16.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.465+0000 7f697bfff640 1 -- 172.21.15.19:0/3623102548 <== mon.1 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v21) v1 ==== 173+0+0 (secure 0 0 0) 0x7f69880426f0 con 0x7f69940b3990 2023-10-06T13:53:16.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.465+0000 7f697bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2023-10-06T13:53:16.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.465+0000 7f697bfff640 10 monclient: _finish_command 2 = system:0 enabled application 'rbd' on pool 'rbd' 2023-10-06T13:53:16.468 INFO:teuthology.orchestra.run.smithi019.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T13:53:16.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 10 librados: watch_flush enter 2023-10-06T13:53:16.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 10 librados: watch_flush exit 2023-10-06T13:53:16.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 20 client.4285.objecter shutdown clearing up homeless session... 2023-10-06T13:53:16.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 10 client.4285.objecter successfully canceled tick 2023-10-06T13:53:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 1 -- 172.21.15.19:0/3623102548 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f698006c180 msgr2=0x7f698006e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 1 --2- 172.21.15.19:0/3623102548 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f698006c180 0x7f698006e630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6984007fd0 tx=0x7f6984007e80 comp rx=0 tx=0).stop 2023-10-06T13:53:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 10 monclient: shutdown 2023-10-06T13:53:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 1 -- 172.21.15.19:0/3623102548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b3990 msgr2=0x7f6994143de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 1 --2- 172.21.15.19:0/3623102548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b3990 0x7f6994143de0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f698800d5d0 tx=0x7f698800da90 comp rx=0 tx=0).stop 2023-10-06T13:53:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 1 -- 172.21.15.19:0/3623102548 shutdown_connections 2023-10-06T13:53:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 1 --2- 172.21.15.19:0/3623102548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69940b3d60 0x7f699413a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 1 --2- 172.21.15.19:0/3623102548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b3990 0x7f6994143de0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 1 --2- 172.21.15.19:0/3623102548 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f698006c180 0x7f698006e630 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 1 --2- 172.21.15.19:0/3623102548 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f69940b00c0 0x7f69940aa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:16.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 1 -- 172.21.15.19:0/3623102548 >> 172.21.15.19:0/3623102548 conn(0x7f69940b6e00 msgr2=0x7f699413e490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:16.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 1 -- 172.21.15.19:0/3623102548 shutdown_connections 2023-10-06T13:53:16.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 1 -- 172.21.15.19:0/3623102548 wait complete. 2023-10-06T13:53:16.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.468+0000 7f6999bd0640 1 librados: shutdown 2023-10-06T13:53:16.485 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T13:53:16.485 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T13:53:16.485 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T13:53:16.486 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T13:53:16.486 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T13:53:16.593 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.591+0000 7f5e4ef00640 10 monclient: get_monmap_and_config 2023-10-06T13:53:16.593 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.591+0000 7f5e4ef00640 10 monclient: build_initial_monmap 2023-10-06T13:53:16.593 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.591+0000 7f5e4ef00640 10 monclient: monmap: 2023-10-06T13:53:16.593 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:16.593 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:16.594 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:16.592645+0000 2023-10-06T13:53:16.594 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:16.592645+0000 2023-10-06T13:53:16.594 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:16.594 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:16.594 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:16.594 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:16.594 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:16.594 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:16.594 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:16.594 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:16.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.592+0000 7f5e4ef00640 1 Processor -- start 2023-10-06T13:53:16.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.592+0000 7f5e4ef00640 1 -- start start 2023-10-06T13:53:16.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.592+0000 7f5e4ef00640 10 monclient: init 2023-10-06T13:53:16.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.592+0000 7f5e4ef00640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:16.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.592+0000 7f5e4ef00640 10 monclient: _add_conns ranks=[0,1,4,2,3] 2023-10-06T13:53:16.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.592+0000 7f5e4ef00640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5e4806b6d0 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:16.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.592+0000 7f5e4ef00640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5e48144910 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:16.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.593+0000 7f5e4ef00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5e4813e630 0x7f5e4814c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:16.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.593+0000 7f5e4ef00640 10 monclient(hunting): picked mon.noname-c con 0x7f5e4813e630 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:16.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.593+0000 7f5e4ef00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4806baa0 con 0x7f5e4806b6d0 2023-10-06T13:53:16.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.593+0000 7f5e4ef00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e480b2530 con 0x7f5e48144910 2023-10-06T13:53:16.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.593+0000 7f5e4ef00640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:16.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.593+0000 7f5e4ef00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e480b2810 con 0x7f5e4813e630 2023-10-06T13:53:16.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.593+0000 7f5e4ef00640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:16.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.593+0000 7f5e4ef00640 10 monclient(hunting): authenticate will time out at 1324.779427s 2023-10-06T13:53:16.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.593+0000 7f5e4cc9c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5e4806b6d0 0x7f5e48144200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57850/0 (socket says 172.21.15.19:57850) 2023-10-06T13:53:16.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.593+0000 7f5e4cc9c640 1 -- 172.21.15.19:0/1900467498 learned_addr learned my addr 172.21.15.19:0/1900467498 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:16.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.593+0000 7f5e4d49d640 1 --2- 172.21.15.19:0/1900467498 >> v2:172.21.15.83:3300/0 conn(0x7f5e4813e630 0x7f5e4814c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:16.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.594+0000 7f5e4d49d640 10 monclient(hunting): get_auth_request con 0x7f5e4813e630 auth_method 0 2023-10-06T13:53:16.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.594+0000 7f5e4d49d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:16.598 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.594+0000 7f5e4d49d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:16.598 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.594+0000 7f5e4d49d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.595+0000 7f5e4d49d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.595+0000 7f5e4d49d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.595+0000 7f5e4d49d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.596+0000 7f5e477fe640 1 -- 172.21.15.19:0/1900467498 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416614220 0 0) 0x7f5e4806baa0 con 0x7f5e4806b6d0 2023-10-06T13:53:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.596+0000 7f5e477fe640 10 monclient(hunting): _negotiate 2023-10-06T13:53:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.596+0000 7f5e477fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.596+0000 7f5e477fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.596+0000 7f5e477fe640 10 monclient(hunting): my global_id is 4248 2023-10-06T13:53:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.596+0000 7f5e477fe640 1 -- 172.21.15.19:0/1900467498 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e38003520 con 0x7f5e4806b6d0 2023-10-06T13:53:16.600 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.596+0000 7f5e4d49d640 10 monclient(hunting): handle_auth_done global_id 4288 payload 274 2023-10-06T13:53:16.600 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.596+0000 7f5e4d49d640 1 -- 172.21.15.19:0/1900467498 >> v1:172.21.15.83:6789/0 conn(0x7f5e48144910 legacy=0x7f5e4813df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:16.600 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.596+0000 7f5e4d49d640 1 -- 172.21.15.19:0/1900467498 >> v1:172.21.15.19:6789/0 conn(0x7f5e4806b6d0 legacy=0x7f5e48144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:16.600 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.596+0000 7f5e4d49d640 10 monclient: _finish_hunting 0 2023-10-06T13:53:16.600 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.596+0000 7f5e4d49d640 1 monclient: found mon.noname-c 2023-10-06T13:53:16.600 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.596+0000 7f5e4d49d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:16.600 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.596+0000 7f5e4d49d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:53:16.600 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.596+0000 7f5e4d49d640 1 -- 172.21.15.19:0/1900467498 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e480aeff0 con 0x7f5e4813e630 2023-10-06T13:53:16.602 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.600+0000 7f5e4d49d640 1 --2- 172.21.15.19:0/1900467498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e4813e630 0x7f5e4814c540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009d90 tx=0x7f5e4002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35edd20cc93c1960 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e477fe640 1 -- 172.21.15.19:0/1900467498 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4002cd00 con 0x7f5e4813e630 2023-10-06T13:53:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e477fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e477fe640 10 monclient: dump: 2023-10-06T13:53:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:16.607 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:16.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e477fe640 10 monclient: _finish_auth 0 2023-10-06T13:53:16.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e477fe640 10 monclient: _check_auth_tickets 2023-10-06T13:53:16.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:16.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e477fe640 1 -- 172.21.15.19:0/1900467498 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4002ce60 con 0x7f5e4813e630 2023-10-06T13:53:16.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e477fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:16.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e477fe640 1 -- 172.21.15.19:0/1900467498 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40034660 con 0x7f5e4813e630 2023-10-06T13:53:16.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e477fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e477fe640 10 monclient: dump: 2023-10-06T13:53:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:16.610 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e4ef00640 5 monclient: authenticate success, global_id 4288 2023-10-06T13:53:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e4ef00640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.604+0000 7f5e4ef00640 4 set_mon_vals no callback set 2023-10-06T13:53:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.605+0000 7f5e4ef00640 10 monclient: shutdown 2023-10-06T13:53:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.605+0000 7f5e4ef00640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.605+0000 7f5e4ef00640 1 -- 172.21.15.19:0/1900467498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e4813e630 msgr2=0x7f5e4814c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.605+0000 7f5e4ef00640 1 --2- 172.21.15.19:0/1900467498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e4813e630 0x7f5e4814c540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009d90 tx=0x7f5e4002bf50 comp rx=0 tx=0).stop 2023-10-06T13:53:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.605+0000 7f5e4ef00640 1 -- 172.21.15.19:0/1900467498 shutdown_connections 2023-10-06T13:53:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.605+0000 7f5e4ef00640 1 --2- 172.21.15.19:0/1900467498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e4813e630 0x7f5e4814c540 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.605+0000 7f5e4ef00640 1 -- 172.21.15.19:0/1900467498 >> 172.21.15.19:0/1900467498 conn(0x7f5e480665b0 msgr2=0x7f5e4806ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.605+0000 7f5e4ef00640 1 -- 172.21.15.19:0/1900467498 shutdown_connections 2023-10-06T13:53:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.605+0000 7f5e4ef00640 1 -- 172.21.15.19:0/1900467498 wait complete. 2023-10-06T13:53:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.606+0000 7f5e4ef00640 10 monclient: build_initial_monmap 2023-10-06T13:53:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.606+0000 7f5e4ef00640 10 monclient: monmap: 2023-10-06T13:53:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:16.615 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:16.615 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:16.615 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:16.615 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:16.615 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:16.615 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:16.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.606+0000 7f5e4ef00640 1 librados: starting msgr at 2023-10-06T13:53:16.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.606+0000 7f5e4ef00640 1 librados: starting objecter 2023-10-06T13:53:16.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.606+0000 7f5e4ef00640 1 Processor -- start 2023-10-06T13:53:16.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.606+0000 7f5e4ef00640 1 -- start start 2023-10-06T13:53:16.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.606+0000 7f5e4ef00640 1 librados: setting wanted keys 2023-10-06T13:53:16.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.606+0000 7f5e4ef00640 1 librados: calling monclient init 2023-10-06T13:53:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.606+0000 7f5e4ef00640 10 monclient: init 2023-10-06T13:53:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T13:53:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e4806b6d0 0x7f5e481d8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 10 monclient(hunting): picked mon.noname-b con 0x7f5e4806b6d0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e4813e630 0x7f5e480aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 10 monclient(hunting): picked mon.noname-a con 0x7f5e4813e630 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5e48144910 0x7f5e48137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 10 monclient(hunting): picked mon.noname-c con 0x7f5e48144910 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e481d9450 con 0x7f5e4813e630 2023-10-06T13:53:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e48140e60 con 0x7f5e4806b6d0 2023-10-06T13:53:16.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:16.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e48141140 con 0x7f5e48144910 2023-10-06T13:53:16.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:16.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.607+0000 7f5e4ef00640 10 monclient(hunting): authenticate will time out at 1324.794011s 2023-10-06T13:53:16.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e4cc9c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e4806b6d0 0x7f5e481d8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:16.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e47fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e4813e630 0x7f5e480aaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:16.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e4d49d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5e48144910 0x7f5e48137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:16.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e4cc9c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e4806b6d0 0x7f5e481d8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39216/0 (socket says 172.21.15.19:39216) 2023-10-06T13:53:16.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e4cc9c640 1 -- 172.21.15.19:0/3513858820 learned_addr learned my addr 172.21.15.19:0/3513858820 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:16.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e4cc9c640 10 monclient(hunting): get_auth_request con 0x7f5e4806b6d0 auth_method 0 2023-10-06T13:53:16.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e4cc9c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:16.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e4cc9c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e4cc9c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e47fff640 10 monclient(hunting): get_auth_request con 0x7f5e4813e630 auth_method 0 2023-10-06T13:53:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e47fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e47fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e4d49d640 10 monclient(hunting): get_auth_request con 0x7f5e48144910 auth_method 0 2023-10-06T13:53:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e4d49d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e4d49d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.608+0000 7f5e4d49d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.609+0000 7f5e4cc9c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.609+0000 7f5e4cc9c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.609+0000 7f5e4cc9c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.609+0000 7f5e47fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.609+0000 7f5e47fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.609+0000 7f5e47fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.609+0000 7f5e4d49d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.609+0000 7f5e4d49d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.609+0000 7f5e4d49d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:16.620 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.613+0000 7f5e47fff640 10 monclient(hunting): handle_auth_done global_id 4251 payload 824 2023-10-06T13:53:16.620 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.613+0000 7f5e47fff640 1 -- 172.21.15.19:0/3513858820 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5e48144910 msgr2=0x7f5e48137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:16.620 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.613+0000 7f5e47fff640 1 --2- 172.21.15.19:0/3513858820 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5e48144910 0x7f5e48137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:16.620 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.613+0000 7f5e47fff640 1 -- 172.21.15.19:0/3513858820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e4806b6d0 msgr2=0x7f5e481d8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.613+0000 7f5e47fff640 1 --2- 172.21.15.19:0/3513858820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e4806b6d0 0x7f5e481d8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.613+0000 7f5e47fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.613+0000 7f5e47fff640 1 monclient: found mon.noname-a 2023-10-06T13:53:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.613+0000 7f5e47fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.613+0000 7f5e47fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.613+0000 7f5e47fff640 1 -- 172.21.15.19:0/3513858820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e40009a70 con 0x7f5e4813e630 2023-10-06T13:53:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.613+0000 7f5e4cc9c640 1 --2- 172.21.15.19:0/3513858820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e4806b6d0 0x7f5e481d8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T13:53:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.616+0000 7f5e47fff640 1 --2- 172.21.15.19:0/3513858820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e4813e630 0x7f5e480aaa60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400c390 tx=0x7f5e3400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.616+0000 7f5e457fa640 10 client.?.objecter ms_handle_connect 0x7f5e4813e630 2023-10-06T13:53:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.616+0000 7f5e457fa640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e457fa640 1 -- 172.21.15.19:0/3513858820 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e340052d0 con 0x7f5e4813e630 2023-10-06T13:53:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e457fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e457fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e457fa640 10 monclient: dump: 2023-10-06T13:53:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:16.624 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e457fa640 10 monclient: _finish_auth 0 2023-10-06T13:53:16.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e457fa640 10 monclient: _check_auth_tickets 2023-10-06T13:53:16.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e457fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:16.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e457fa640 1 -- 172.21.15.19:0/3513858820 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e34005430 con 0x7f5e4813e630 2023-10-06T13:53:16.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e4ef00640 5 monclient: authenticate success, global_id 4251 2023-10-06T13:53:16.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e4ef00640 10 monclient: _renew_subs 2023-10-06T13:53:16.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e4ef00640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:16.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e4ef00640 1 -- 172.21.15.19:0/3513858820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e480ac8a0 con 0x7f5e4813e630 2023-10-06T13:53:16.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e457fa640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:16.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e457fa640 1 -- 172.21.15.19:0/3513858820 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34003830 con 0x7f5e4813e630 2023-10-06T13:53:16.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e457fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:16.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e457fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:16.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e457fa640 10 monclient: dump: 2023-10-06T13:53:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:16.626 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e46ffd640 4 set_mon_vals no callback set 2023-10-06T13:53:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e4ef00640 10 client.4251.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e4ef00640 10 monclient: _renew_subs 2023-10-06T13:53:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e4ef00640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e4ef00640 1 -- 172.21.15.19:0/3513858820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e481d9ae0 con 0x7f5e4813e630 2023-10-06T13:53:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.621+0000 7f5e4ef00640 1 librados: init done 2023-10-06T13:53:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.622+0000 7f5e4ef00640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.622+0000 7f5e4ef00640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.622+0000 7f5e4ef00640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:16.628 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.622+0000 7f5e4ef00640 1 -- 172.21.15.19:0/3513858820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e481d9620 con 0x7f5e4813e630 2023-10-06T13:53:16.644 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.643+0000 7f5e457fa640 1 -- 172.21.15.19:0/3513858820 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5e34003990 con 0x7f5e4813e630 2023-10-06T13:53:16.645 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.643+0000 7f5e457fa640 1 --2- 172.21.15.19:0/3513858820 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5e2c06c1c0 0x7f5e2c06e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:16.645 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.643+0000 7f5e4cc9c640 1 --2- 172.21.15.19:0/3513858820 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5e2c06c1c0 0x7f5e2c06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:16.645 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.643+0000 7f5e4cc9c640 10 monclient: get_auth_request con 0x7f5e2c06c1c0 auth_method 0 2023-10-06T13:53:16.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.644+0000 7f5e4cc9c640 1 --2- 172.21.15.19:0/3513858820 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5e2c06c1c0 0x7f5e2c06e670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0011e0 tx=0x7f5e3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:16.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.644+0000 7f5e457fa640 10 client.4251.objecter ms_handle_connect 0x7f5e2c06c1c0 2023-10-06T13:53:16.648 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.646+0000 7f5e457fa640 1 -- 172.21.15.19:0/3513858820 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5e340576c0 con 0x7f5e4813e630 2023-10-06T13:53:16.648 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.646+0000 7f5e457fa640 10 client.4251.objecter ms_dispatch 0x7f5e48000b90 osd_map(21..21 src has 1..21) v4 2023-10-06T13:53:16.648 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.646+0000 7f5e457fa640 3 client.4251.objecter handle_osd_map got epochs [21,21] > 0 2023-10-06T13:53:16.648 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.646+0000 7f5e457fa640 3 client.4251.objecter handle_osd_map decoding full epoch 21 2023-10-06T13:53:16.648 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.646+0000 7f5e457fa640 20 client.4251.objecter dump_active .. 0 homeless 2023-10-06T13:53:16.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.956+0000 7f5e457fa640 1 -- 172.21.15.19:0/3513858820 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f5e34053a00 con 0x7f5e4813e630 2023-10-06T13:53:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.956+0000 7f5e457fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:16.956+0000 7f5e457fa640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:17.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.200+0000 7f5e4ef00640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2023-10-06T13:53:17.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.200+0000 7f5e4ef00640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2023-10-06T13:53:17.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.200+0000 7f5e4ef00640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:17.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.200+0000 7f5e4ef00640 1 -- 172.21.15.19:0/3513858820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5e481d9620 con 0x7f5e4813e630 2023-10-06T13:53:17.400 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.398+0000 7f5e457fa640 1 -- 172.21.15.19:0/3513858820 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170488 (secure 0 0 0) 0x7f5e481d9620 con 0x7f5e4813e630 2023-10-06T13:53:17.400 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.398+0000 7f5e457fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2023-10-06T13:53:17.400 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.398+0000 7f5e457fa640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:17.402 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:53:17.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.400+0000 7f5e4ef00640 10 librados: watch_flush enter 2023-10-06T13:53:17.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.400+0000 7f5e4ef00640 10 librados: watch_flush exit 2023-10-06T13:53:17.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.400+0000 7f5e4ef00640 20 client.4251.objecter shutdown clearing up homeless session... 2023-10-06T13:53:17.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.400+0000 7f5e4ef00640 10 client.4251.objecter successfully canceled tick 2023-10-06T13:53:17.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.400+0000 7f5e4ef00640 1 -- 172.21.15.19:0/3513858820 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5e2c06c1c0 msgr2=0x7f5e2c06e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:17.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.400+0000 7f5e4ef00640 1 --2- 172.21.15.19:0/3513858820 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5e2c06c1c0 0x7f5e2c06e670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0011e0 tx=0x7f5e3c009040 comp rx=0 tx=0).stop 2023-10-06T13:53:17.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.400+0000 7f5e4ef00640 10 monclient: shutdown 2023-10-06T13:53:17.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.400+0000 7f5e4ef00640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:17.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.400+0000 7f5e4ef00640 1 -- 172.21.15.19:0/3513858820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e4813e630 msgr2=0x7f5e480aaa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:17.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.400+0000 7f5e4ef00640 1 --2- 172.21.15.19:0/3513858820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e4813e630 0x7f5e480aaa60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400c390 tx=0x7f5e3400c850 comp rx=0 tx=0).stop 2023-10-06T13:53:17.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.401+0000 7f5e4ef00640 1 -- 172.21.15.19:0/3513858820 shutdown_connections 2023-10-06T13:53:17.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.401+0000 7f5e4ef00640 1 --2- 172.21.15.19:0/3513858820 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5e2c06c1c0 0x7f5e2c06e670 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:17.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.401+0000 7f5e4ef00640 1 --2- 172.21.15.19:0/3513858820 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5e48144910 0x7f5e48137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.401+0000 7f5e4ef00640 1 --2- 172.21.15.19:0/3513858820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e4813e630 0x7f5e480aaa60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.401+0000 7f5e4ef00640 1 --2- 172.21.15.19:0/3513858820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e4806b6d0 0x7f5e481d8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.401+0000 7f5e4ef00640 1 -- 172.21.15.19:0/3513858820 >> 172.21.15.19:0/3513858820 conn(0x7f5e480665b0 msgr2=0x7f5e480af710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.401+0000 7f5e4ef00640 1 -- 172.21.15.19:0/3513858820 shutdown_connections 2023-10-06T13:53:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.401+0000 7f5e4ef00640 1 -- 172.21.15.19:0/3513858820 wait complete. 2023-10-06T13:53:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.401+0000 7f5e4ef00640 1 librados: shutdown 2023-10-06T13:53:17.413 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":5,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":85581},{"type":"v1","addr":"172.21.15.83:6801","nonce":85581}]},"active_addr":"172.21.15.83:6801/85581","active_change":"2023-10-06T13:52:25.019133+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":392130776}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3566033827}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":506837463}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":836031698}]}]} 2023-10-06T13:53:17.418 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T13:53:17.418 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T13:53:17.418 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T13:53:17.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.529+0000 7fd46c577640 10 monclient: get_monmap_and_config 2023-10-06T13:53:17.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.529+0000 7fd46c577640 10 monclient: build_initial_monmap 2023-10-06T13:53:17.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.529+0000 7fd46c577640 10 monclient: monmap: 2023-10-06T13:53:17.531 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:17.531 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:17.531 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:17.530134+0000 2023-10-06T13:53:17.531 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:17.530134+0000 2023-10-06T13:53:17.531 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:17.531 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:17.532 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:17.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.529+0000 7fd46c577640 1 Processor -- start 2023-10-06T13:53:17.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.529+0000 7fd46c577640 1 -- start start 2023-10-06T13:53:17.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.529+0000 7fd46c577640 10 monclient: init 2023-10-06T13:53:17.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.530+0000 7fd46c577640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:17.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.530+0000 7fd46c577640 10 monclient: _add_conns ranks=[0,3,2,1,4] 2023-10-06T13:53:17.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.530+0000 7fd46c577640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd46406b6d0 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:17.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.530+0000 7fd46c577640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd464144910 0x7fd46413df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:17.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.530+0000 7fd46c577640 10 monclient(hunting): picked mon.noname-a con 0x7fd464144910 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:17.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.530+0000 7fd46c577640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd46413e460 0x7fd46414b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:17.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.530+0000 7fd46c577640 10 monclient(hunting): picked mon.noname-b con 0x7fd46413e460 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:17.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.530+0000 7fd46c577640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46406baa0 con 0x7fd46406b6d0 2023-10-06T13:53:17.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.530+0000 7fd46c577640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.530+0000 7fd46c577640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4640b2530 con 0x7fd464144910 2023-10-06T13:53:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.530+0000 7fd46c577640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.530+0000 7fd46c577640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4640b2810 con 0x7fd46413e460 2023-10-06T13:53:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.530+0000 7fd46c577640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.530+0000 7fd46c577640 10 monclient(hunting): authenticate will time out at 1325.716872s 2023-10-06T13:53:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.531+0000 7fd46ab14640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd46413e460 0x7fd46414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.531+0000 7fd46ab14640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd46413e460 0x7fd46414b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:41216/0 (socket says 172.21.15.19:41216) 2023-10-06T13:53:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.531+0000 7fd46ab14640 1 -- 172.21.15.19:0/4120170875 learned_addr learned my addr 172.21.15.19:0/4120170875 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.531+0000 7fd46ab14640 10 monclient(hunting): get_auth_request con 0x7fd46413e460 auth_method 0 2023-10-06T13:53:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.531+0000 7fd46ab14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.531+0000 7fd46ab14640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:17.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.531+0000 7fd46ab14640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.532+0000 7fd46ab14640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.532+0000 7fd46ab14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.532+0000 7fd46ab14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:17.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.533+0000 7fd46ab14640 10 monclient(hunting): handle_auth_done global_id 4226 payload 274 2023-10-06T13:53:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.533+0000 7fd46ab14640 1 -- 172.21.15.19:0/4120170875 >> v2:172.21.15.19:3300/0 conn(0x7fd464144910 msgr2=0x7fd46413df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.533+0000 7fd46ab14640 1 --2- 172.21.15.19:0/4120170875 >> v2:172.21.15.19:3300/0 conn(0x7fd464144910 0x7fd46413df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.533+0000 7fd46ab14640 1 -- 172.21.15.19:0/4120170875 >> v1:172.21.15.19:6789/0 conn(0x7fd46406b6d0 legacy=0x7fd464144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.533+0000 7fd46ab14640 10 monclient: _finish_hunting 0 2023-10-06T13:53:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.533+0000 7fd46ab14640 1 monclient: found mon.noname-b 2023-10-06T13:53:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.533+0000 7fd46ab14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.533+0000 7fd46ab14640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T13:53:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.533+0000 7fd46ab14640 1 -- 172.21.15.19:0/4120170875 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4640aeff0 con 0x7fd46413e460 2023-10-06T13:53:17.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.535+0000 7fd46ab14640 1 --2- 172.21.15.19:0/4120170875 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd46413e460 0x7fd46414b530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009e50 tx=0x7fd45c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d67da12449c9ae8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:17.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd469311640 1 -- 172.21.15.19:0/4120170875 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c02ca70 con 0x7fd46413e460 2023-10-06T13:53:17.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd469311640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd469311640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd469311640 10 monclient: dump: 2023-10-06T13:53:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:17.543 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd469311640 10 monclient: _finish_auth 0 2023-10-06T13:53:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd469311640 10 monclient: _check_auth_tickets 2023-10-06T13:53:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd469311640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd469311640 1 -- 172.21.15.19:0/4120170875 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c02cbd0 con 0x7fd46413e460 2023-10-06T13:53:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd46c577640 5 monclient: authenticate success, global_id 4226 2023-10-06T13:53:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd46c577640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd469311640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd46c577640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd46c577640 4 set_mon_vals no callback set 2023-10-06T13:53:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd469311640 1 -- 172.21.15.19:0/4120170875 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0343f0 con 0x7fd46413e460 2023-10-06T13:53:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd469311640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd469311640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.539+0000 7fd469311640 10 monclient: dump: 2023-10-06T13:53:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:17.544 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:17.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.540+0000 7fd46c577640 10 monclient: shutdown 2023-10-06T13:53:17.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.540+0000 7fd46c577640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:17.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.540+0000 7fd46c577640 1 -- 172.21.15.19:0/4120170875 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd46413e460 msgr2=0x7fd46414b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:17.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.540+0000 7fd46c577640 1 --2- 172.21.15.19:0/4120170875 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd46413e460 0x7fd46414b530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009e50 tx=0x7fd45c02bed0 comp rx=0 tx=0).stop 2023-10-06T13:53:17.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.540+0000 7fd46c577640 1 -- 172.21.15.19:0/4120170875 shutdown_connections 2023-10-06T13:53:17.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.540+0000 7fd46c577640 1 --2- 172.21.15.19:0/4120170875 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd46413e460 0x7fd46414b530 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:17.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.540+0000 7fd46c577640 1 --2- 172.21.15.19:0/4120170875 >> v2:172.21.15.19:3300/0 conn(0x7fd464144910 0x7fd46413df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:17.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.540+0000 7fd46c577640 1 -- 172.21.15.19:0/4120170875 >> 172.21.15.19:0/4120170875 conn(0x7fd4640665b0 msgr2=0x7fd46406ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:17.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.540+0000 7fd46c577640 1 -- 172.21.15.19:0/4120170875 shutdown_connections 2023-10-06T13:53:17.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.540+0000 7fd46c577640 1 -- 172.21.15.19:0/4120170875 wait complete. 2023-10-06T13:53:17.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.541+0000 7fd46c577640 10 monclient: build_initial_monmap 2023-10-06T13:53:17.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.541+0000 7fd46c577640 10 monclient: monmap: 2023-10-06T13:53:17.548 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:17.549 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:17.549 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:17.549 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:17.549 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:17.549 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:17.549 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:17.549 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:17.549 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:17.549 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:17.549 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.542+0000 7fd46c577640 1 librados: starting msgr at 2023-10-06T13:53:17.549 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.542+0000 7fd46c577640 1 librados: starting objecter 2023-10-06T13:53:17.549 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.542+0000 7fd46c577640 1 Processor -- start 2023-10-06T13:53:17.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.542+0000 7fd46c577640 1 -- start start 2023-10-06T13:53:17.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.542+0000 7fd46c577640 1 librados: setting wanted keys 2023-10-06T13:53:17.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.542+0000 7fd46c577640 1 librados: calling monclient init 2023-10-06T13:53:17.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.542+0000 7fd46c577640 10 monclient: init 2023-10-06T13:53:17.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:17.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 10 monclient: _add_conns ranks=[2,0,1] 2023-10-06T13:53:17.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd46406b6d0 0x7fd464142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:17.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 10 monclient(hunting): picked mon.noname-c con 0x7fd46406b6d0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:17.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd46413e460 0x7fd4641430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:17.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 10 monclient(hunting): picked mon.noname-a con 0x7fd46413e460 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:17.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464144910 0x7fd4641d6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:17.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 10 monclient(hunting): picked mon.noname-b con 0x7fd464144910 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:17.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:17.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46413f300 con 0x7fd46413e460 2023-10-06T13:53:17.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:17.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd464143720 con 0x7fd464144910 2023-10-06T13:53:17.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:17.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd464143a00 con 0x7fd46406b6d0 2023-10-06T13:53:17.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:17.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46c577640 10 monclient(hunting): authenticate will time out at 1325.729574s 2023-10-06T13:53:17.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46a313640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd46406b6d0 0x7fd464142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:17.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.543+0000 7fd46ab14640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464144910 0x7fd4641d6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:17.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46ab14640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464144910 0x7fd4641d6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39228/0 (socket says 172.21.15.19:39228) 2023-10-06T13:53:17.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46ab14640 1 -- 172.21.15.19:0/362682380 learned_addr learned my addr 172.21.15.19:0/362682380 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46ab14640 10 monclient(hunting): get_auth_request con 0x7fd464144910 auth_method 0 2023-10-06T13:53:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46ab14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46ab14640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46ab14640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46a313640 10 monclient(hunting): get_auth_request con 0x7fd46406b6d0 auth_method 0 2023-10-06T13:53:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46a313640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46a313640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46a313640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46ab14640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46ab14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46ab14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46a313640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46a313640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:17.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.544+0000 7fd46a313640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:17.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.547+0000 7fd46ab14640 10 monclient(hunting): handle_auth_done global_id 4297 payload 824 2023-10-06T13:53:17.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.547+0000 7fd46ab14640 1 -- 172.21.15.19:0/362682380 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd46406b6d0 msgr2=0x7fd464142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:17.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.547+0000 7fd46ab14640 1 --2- 172.21.15.19:0/362682380 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd46406b6d0 0x7fd464142ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:17.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.547+0000 7fd46ab14640 1 -- 172.21.15.19:0/362682380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd46413e460 msgr2=0x7fd4641430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:17.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.547+0000 7fd46ab14640 1 --2- 172.21.15.19:0/362682380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd46413e460 0x7fd4641430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:17.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.547+0000 7fd46ab14640 10 monclient: _finish_hunting 0 2023-10-06T13:53:17.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.547+0000 7fd46ab14640 1 monclient: found mon.noname-b 2023-10-06T13:53:17.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.547+0000 7fd46ab14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:17.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.547+0000 7fd46ab14640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:17.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.547+0000 7fd46ab14640 1 -- 172.21.15.19:0/362682380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c009b30 con 0x7fd464144910 2023-10-06T13:53:17.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.549+0000 7fd46ab14640 1 --2- 172.21.15.19:0/362682380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464144910 0x7fd4641d6060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd45c036040 tx=0x7fd45c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:17.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.549+0000 7fd45affd640 10 client.?.objecter ms_handle_connect 0x7fd464144910 2023-10-06T13:53:17.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.549+0000 7fd45affd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:17.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd45affd640 1 -- 172.21.15.19:0/362682380 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c03e450 con 0x7fd464144910 2023-10-06T13:53:17.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd45affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:17.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd45affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:17.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd45affd640 10 monclient: dump: 2023-10-06T13:53:17.558 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:17.558 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:17.558 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd45affd640 10 monclient: _finish_auth 0 2023-10-06T13:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd45affd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd45affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd45affd640 1 -- 172.21.15.19:0/362682380 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c03e5d0 con 0x7fd464144910 2023-10-06T13:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd45affd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd45affd640 1 -- 172.21.15.19:0/362682380 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c03e980 con 0x7fd464144910 2023-10-06T13:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd46c577640 5 monclient: authenticate success, global_id 4297 2023-10-06T13:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd46c577640 10 monclient: _renew_subs 2023-10-06T13:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd46c577640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd46c577640 1 -- 172.21.15.19:0/362682380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd464143c80 con 0x7fd464144910 2023-10-06T13:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd468b10640 4 set_mon_vals no callback set 2023-10-06T13:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd45affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd45affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd45affd640 10 monclient: dump: 2023-10-06T13:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd46c577640 10 client.4297.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd46c577640 10 monclient: _renew_subs 2023-10-06T13:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd46c577640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd46c577640 1 -- 172.21.15.19:0/362682380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4641d6780 con 0x7fd464144910 2023-10-06T13:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.553+0000 7fd46c577640 1 librados: init done 2023-10-06T13:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.554+0000 7fd46c577640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.554+0000 7fd46c577640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:17.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.554+0000 7fd46c577640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:17.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.554+0000 7fd46c577640 1 -- 172.21.15.19:0/362682380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4640b2c80 con 0x7fd464144910 2023-10-06T13:53:17.573 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.571+0000 7fd45affd640 1 -- 172.21.15.19:0/362682380 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd45c03eae0 con 0x7fd464144910 2023-10-06T13:53:17.573 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.571+0000 7fd45affd640 1 --2- 172.21.15.19:0/362682380 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd44806c210 0x7fd44806e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:17.573 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.572+0000 7fd46a313640 1 --2- 172.21.15.19:0/362682380 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd44806c210 0x7fd44806e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:17.574 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.572+0000 7fd46a313640 10 monclient: get_auth_request con 0x7fd44806c210 auth_method 0 2023-10-06T13:53:17.574 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.572+0000 7fd46a313640 1 --2- 172.21.15.19:0/362682380 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd44806c210 0x7fd44806e6c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd460006fd0 tx=0x7fd460009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:17.574 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.572+0000 7fd45affd640 10 client.4297.objecter ms_handle_connect 0x7fd44806c210 2023-10-06T13:53:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.574+0000 7fd45affd640 1 -- 172.21.15.19:0/362682380 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd45c07b720 con 0x7fd464144910 2023-10-06T13:53:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.574+0000 7fd45affd640 10 client.4297.objecter ms_dispatch 0x7fd464000b90 osd_map(21..21 src has 1..21) v4 2023-10-06T13:53:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.574+0000 7fd45affd640 3 client.4297.objecter handle_osd_map got epochs [21,21] > 0 2023-10-06T13:53:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.574+0000 7fd45affd640 3 client.4297.objecter handle_osd_map decoding full epoch 21 2023-10-06T13:53:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.574+0000 7fd45affd640 20 client.4297.objecter dump_active .. 0 homeless 2023-10-06T13:53:17.621 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:01:20.744 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:17.629 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:01:20.752 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:17.698 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:01:20.821 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:17.734 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:01:20.857 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:17.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.811+0000 7fd45affd640 1 -- 172.21.15.19:0/362682380 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7fd45c077360 con 0x7fd464144910 2023-10-06T13:53:17.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.811+0000 7fd45affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:17.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.811+0000 7fd45affd640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:17.916 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:01:21.013 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:17.972 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:01:21.069 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:17.973 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:01:21.096 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:17.990 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:01:21.113 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.992+0000 7fd46c577640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:53:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.992+0000 7fd46c577640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:53:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.992+0000 7fd46c577640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:17.992+0000 7fd46c577640 1 -- 172.21.15.19:0/362682380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4641d5d80 con 0x7fd464144910 2023-10-06T13:53:18.227 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:01:21.324 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:18.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.323+0000 7fd45affd640 1 -- 172.21.15.19:0/362682380 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+14159 (secure 0 0 0) 0x7fd4641d5d80 con 0x7fd464144910 2023-10-06T13:53:18.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.323+0000 7fd45affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:53:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.323+0000 7fd45affd640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:18.326 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:53:18.326 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"fsid":"c5a90f7c-81ec-43a3-ab47-af6ce5729ec5","created":"2023-10-06T13:52:19.516184+0000","modified":"2023-10-06T13:53:17.207818+0000","last_up_change":"2023-10-06T13:53:08.156870+0000","last_in_change":"2023-10-06T13:52:24.916896+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T13:53:07.784836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T13:53:14.364162+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6cb75c5-86fb-48b9-895c-fb1d72bd4746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":86195},{"type":"v1","addr":"172.21.15.19:6818","nonce":86195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6821","nonce":86195},{"type":"v1","addr":"172.21.15.19:6823","nonce":86195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":86195},{"type":"v1","addr":"172.21.15.19:6831","nonce":86195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":86195},{"type":"v1","addr":"172.21.15.19:6827","nonce":86195}]},"public_addr":"172.21.15.19:6818/86195","cluster_addr":"172.21.15.19:6823/86195","heartbeat_back_addr":"172.21.15.19:6831/86195","heartbeat_front_addr":"172.21.15.19:6827/86195","state":["exists","up"]},{"osd":1,"uuid":"ba828cb4-a4f6-4fea-a02e-24d9cf597ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":86193},{"type":"v1","addr":"172.21.15.19:6819","nonce":86193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":86193},{"type":"v1","addr":"172.21.15.19:6822","nonce":86193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":86193},{"type":"v1","addr":"172.21.15.19:6830","nonce":86193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":86193},{"type":"v1","addr":"172.21.15.19:6826","nonce":86193}]},"public_addr":"172.21.15.19:6819/86193","cluster_addr":"172.21.15.19:6822/86193","heartbeat_back_addr":"172.21.15.19:6830/86193","heartbeat_front_addr":"172.21.15.19:6826/86193","state":["exists","up"]},{"osd":2,"uuid":"06bcfab5-d2f1-4447-ba44-143398707ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86194},{"type":"v1","addr":"172.21.15.19:6809","nonce":86194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":86194},{"type":"v1","addr":"172.21.15.19:6811","nonce":86194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":86194},{"type":"v1","addr":"172.21.15.19:6815","nonce":86194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":86194},{"type":"v1","addr":"172.21.15.19:6813","nonce":86194}]},"public_addr":"172.21.15.19:6809/86194","cluster_addr":"172.21.15.19:6811/86194","heartbeat_back_addr":"172.21.15.19:6815/86194","heartbeat_front_addr":"172.21.15.19:6813/86194","state":["exists","up"]},{"osd":3,"uuid":"7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":86196},{"type":"v1","addr":"172.21.15.19:6801","nonce":86196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86196},{"type":"v1","addr":"172.21.15.19:6803","nonce":86196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86196},{"type":"v1","addr":"172.21.15.19:6807","nonce":86196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86196},{"type":"v1","addr":"172.21.15.19:6805","nonce":86196}]},"public_addr":"172.21.15.19:6801/86196","cluster_addr":"172.21.15.19:6803/86196","heartbeat_back_addr":"172.21.15.19:6807/86196","heartbeat_front_addr":"172.21.15.19:6805/86196","state":["exists","up"]},{"osd":4,"uuid":"ad31d4df-e78c-4e32-bfd4-3fe6e70dd428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":86123},{"type":"v1","addr":"172.21.15.83:6827","nonce":86123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":86123},{"type":"v1","addr":"172.21.15.83:6829","nonce":86123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":86123},{"type":"v1","addr":"172.21.15.83:6833","nonce":86123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":86123},{"type":"v1","addr":"172.21.15.83:6831","nonce":86123}]},"public_addr":"172.21.15.83:6827/86123","cluster_addr":"172.21.15.83:6829/86123","heartbeat_back_addr":"172.21.15.83:6833/86123","heartbeat_front_addr":"172.21.15.83:6831/86123","state":["exists","up"]},{"osd":5,"uuid":"931b23f8-80c3-4595-8931-2568be479cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":86126},{"type":"v1","addr":"172.21.15.83:6819","nonce":86126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":86126},{"type":"v1","addr":"172.21.15.83:6821","nonce":86126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":86126},{"type":"v1","addr":"172.21.15.83:6825","nonce":86126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":86126},{"type":"v1","addr":"172.21.15.83:6823","nonce":86126}]},"public_addr":"172.21.15.83:6819/86126","cluster_addr":"172.21.15.83:6821/86126","heartbeat_back_addr":"172.21.15.83:6825/86126","heartbeat_front_addr":"172.21.15.83:6823/86126","state":["exists","up"]},{"osd":6,"uuid":"70dd2f95-93eb-448a-a65d-3a09bdea85a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":86124},{"type":"v1","addr":"172.21.15.83:6803","nonce":86124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":86124},{"type":"v1","addr":"172.21.15.83:6805","nonce":86124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":86124},{"type":"v1","addr":"172.21.15.83:6809","nonce":86124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":86124},{"type":"v1","addr":"172.21.15.83:6807","nonce":86124}]},"public_addr":"172.21.15.83:6803/86124","cluster_addr":"172.21.15.83:6805/86124","heartbeat_back_addr":"172.21.15.83:6809/86124","heartbeat_front_addr":"172.21.15.83:6807/86124","state":["exists","up"]},{"osd":7,"uuid":"9797eb84-1b2c-4280-ab13-bdb7b2007dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":86125},{"type":"v1","addr":"172.21.15.83:6811","nonce":86125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":86125},{"type":"v1","addr":"172.21.15.83:6813","nonce":86125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":86125},{"type":"v1","addr":"172.21.15.83:6817","nonce":86125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":86125},{"type":"v1","addr":"172.21.15.83:6815","nonce":86125}]},"public_addr":"172.21.15.83:6811/86125","cluster_addr":"172.21.15.83:6813/86125","heartbeat_back_addr":"172.21.15.83:6817/86125","heartbeat_front_addr":"172.21.15.83:6815/86125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:04.886714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:05.153337+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:04.794593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:04.049373+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:03.567476+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:02.921086+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:02.431856+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:02.863561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T13:53:18.328 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.325+0000 7fd46c577640 10 librados: watch_flush enter 2023-10-06T13:53:18.328 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 10 librados: watch_flush exit 2023-10-06T13:53:18.328 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 20 client.4297.objecter shutdown clearing up homeless session... 2023-10-06T13:53:18.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 10 client.4297.objecter successfully canceled tick 2023-10-06T13:53:18.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 1 -- 172.21.15.19:0/362682380 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd44806c210 msgr2=0x7fd44806e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:18.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 1 --2- 172.21.15.19:0/362682380 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd44806c210 0x7fd44806e6c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd460006fd0 tx=0x7fd460009060 comp rx=0 tx=0).stop 2023-10-06T13:53:18.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 10 monclient: shutdown 2023-10-06T13:53:18.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:18.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 1 -- 172.21.15.19:0/362682380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464144910 msgr2=0x7fd4641d6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:18.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 1 --2- 172.21.15.19:0/362682380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464144910 0x7fd4641d6060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd45c036040 tx=0x7fd45c004a40 comp rx=0 tx=0).stop 2023-10-06T13:53:18.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 1 -- 172.21.15.19:0/362682380 shutdown_connections 2023-10-06T13:53:18.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 1 --2- 172.21.15.19:0/362682380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464144910 0x7fd4641d6060 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 1 --2- 172.21.15.19:0/362682380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd46413e460 0x7fd4641430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 1 --2- 172.21.15.19:0/362682380 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd44806c210 0x7fd44806e6c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 1 --2- 172.21.15.19:0/362682380 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd46406b6d0 0x7fd464142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 1 -- 172.21.15.19:0/362682380 >> 172.21.15.19:0/362682380 conn(0x7fd4640665b0 msgr2=0x7fd46413c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 1 -- 172.21.15.19:0/362682380 shutdown_connections 2023-10-06T13:53:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 1 -- 172.21.15.19:0/362682380 wait complete. 2023-10-06T13:53:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.326+0000 7fd46c577640 1 librados: shutdown 2023-10-06T13:53:18.339 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T13:53:18.339 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T13:53:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.443+0000 7face4a8b640 10 monclient: get_monmap_and_config 2023-10-06T13:53:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.443+0000 7face4a8b640 10 monclient: build_initial_monmap 2023-10-06T13:53:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.443+0000 7face4a8b640 10 monclient: monmap: 2023-10-06T13:53:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:18.444891+0000 2023-10-06T13:53:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:18.444891+0000 2023-10-06T13:53:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:18.447 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:18.447 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:18.447 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:18.448 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.444+0000 7face4a8b640 1 Processor -- start 2023-10-06T13:53:18.448 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.444+0000 7face4a8b640 1 -- start start 2023-10-06T13:53:18.448 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.444+0000 7face4a8b640 10 monclient: init 2023-10-06T13:53:18.448 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.445+0000 7face4a8b640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:18.448 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.445+0000 7face4a8b640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2023-10-06T13:53:18.448 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.445+0000 7face4a8b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7face006b6d0 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:18.448 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.445+0000 7face4a8b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7face0144910 0x7face013df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:18.448 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.445+0000 7face4a8b640 10 monclient(hunting): picked mon.noname-c con 0x7face0144910 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:18.448 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.445+0000 7face4a8b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7face013e460 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:18.449 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.445+0000 7face4a8b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face006baa0 con 0x7face013e460 2023-10-06T13:53:18.449 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.445+0000 7face4a8b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face00b2530 con 0x7face006b6d0 2023-10-06T13:53:18.449 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.445+0000 7face4a8b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:18.449 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.445+0000 7face4a8b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face00b2810 con 0x7face0144910 2023-10-06T13:53:18.449 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.445+0000 7face4a8b640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:18.449 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.445+0000 7face4a8b640 10 monclient(hunting): authenticate will time out at 1326.631612s 2023-10-06T13:53:18.449 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.445+0000 7facded9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7face013e460 0x7face014b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57862/0 (socket says 172.21.15.19:57862) 2023-10-06T13:53:18.449 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.445+0000 7facded9d640 1 -- 172.21.15.19:0/2943273238 learned_addr learned my addr 172.21.15.19:0/2943273238 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:18.449 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.447+0000 7facdd59a640 1 -- 172.21.15.19:0/2943273238 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2274594817 0 0) 0x7face006baa0 con 0x7face013e460 2023-10-06T13:53:18.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.447+0000 7facdd59a640 10 monclient(hunting): _negotiate 2023-10-06T13:53:18.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.447+0000 7facdd59a640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:18.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.447+0000 7facdd59a640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:18.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.447+0000 7facdd59a640 10 monclient(hunting): my global_id is 4254 2023-10-06T13:53:18.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.447+0000 7facdd59a640 1 -- 172.21.15.19:0/2943273238 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faccc003520 con 0x7face013e460 2023-10-06T13:53:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 -- 172.21.15.19:0/2943273238 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4001464115 0 0) 0x7faccc003520 con 0x7face013e460 2023-10-06T13:53:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 -- 172.21.15.19:0/2943273238 >> v2:172.21.15.83:3300/0 conn(0x7face0144910 msgr2=0x7face013df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 --2- 172.21.15.19:0/2943273238 >> v2:172.21.15.83:3300/0 conn(0x7face0144910 0x7face013df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 -- 172.21.15.19:0/2943273238 >> v1:172.21.15.83:6789/0 conn(0x7face006b6d0 legacy=0x7face0144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient: _finish_hunting 0 2023-10-06T13:53:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 monclient: found mon.noname-a-legacy 2023-10-06T13:53:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.19:6789/0 2023-10-06T13:53:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 -- 172.21.15.19:0/2943273238 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face00aeff0 con 0x7face013e460 2023-10-06T13:53:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient: _finish_auth 0 2023-10-06T13:53:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient: _check_auth_tickets 2023-10-06T13:53:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 -- 172.21.15.19:0/2943273238 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251531364 0 0) 0x7facd00019f0 con 0x7face013e460 2023-10-06T13:53:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7face4a8b640 5 monclient: authenticate success, global_id 4254 2023-10-06T13:53:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7face4a8b640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2023-10-06T13:53:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient: dump: 2023-10-06T13:53:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:18.456 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] but i'm connected to v1:172.21.15.19:6789/0, reconnecting 2023-10-06T13:53:18.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:18.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 -- 172.21.15.19:0/2943273238 >> v1:172.21.15.19:6789/0 conn(0x7face013e460 legacy=0x7face014b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:18.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:53:18.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 --2- 172.21.15.19:0/2943273238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faccc0039c0 0x7faccc003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:18.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient(hunting): picked mon.a con 0x7faccc0039c0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:18.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 --2- 172.21.15.19:0/2943273238 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faccc0047c0 0x7faccc004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:18.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient(hunting): picked mon.c con 0x7faccc0047c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:18.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 --2- 172.21.15.19:0/2943273238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faccc005270 0x7faccc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:18.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient(hunting): picked mon.b con 0x7faccc005270 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:18.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:18.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 -- 172.21.15.19:0/2943273238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face00b2810 con 0x7faccc0039c0 2023-10-06T13:53:18.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 -- 172.21.15.19:0/2943273238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc00c1a0 con 0x7faccc005270 2023-10-06T13:53:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 1 -- 172.21.15.19:0/2943273238 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc00c360 con 0x7faccc0047c0 2023-10-06T13:53:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7facdd59a640 10 monclient(hunting): _finish_auth 0 2023-10-06T13:53:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.451+0000 7face4a8b640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.452+0000 7facddd9b640 1 --2- 172.21.15.19:0/2943273238 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faccc0047c0 0x7faccc004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.452+0000 7facddd9b640 10 monclient(hunting): get_auth_request con 0x7faccc0047c0 auth_method 0 2023-10-06T13:53:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.452+0000 7facddd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.452+0000 7facddd9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.452+0000 7facddd9b640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:53:18.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.453+0000 7facddd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:18.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.453+0000 7facddd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:18.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.453+0000 7facddd9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T13:53:18.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.454+0000 7facde59c640 1 --2- 172.21.15.19:0/2943273238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faccc0039c0 0x7faccc003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:18.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.454+0000 7facde59c640 10 monclient(hunting): get_auth_request con 0x7faccc0039c0 auth_method 0 2023-10-06T13:53:18.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.454+0000 7facde59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:18.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.454+0000 7facde59c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:18.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.454+0000 7facde59c640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:53:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.455+0000 7facddd9b640 10 monclient(hunting): handle_auth_done global_id 4254 payload 293 2023-10-06T13:53:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.455+0000 7facddd9b640 1 -- 172.21.15.19:0/2943273238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faccc005270 msgr2=0x7faccc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.455+0000 7facddd9b640 1 --2- 172.21.15.19:0/2943273238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faccc005270 0x7faccc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.455+0000 7facddd9b640 1 -- 172.21.15.19:0/2943273238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faccc0039c0 msgr2=0x7faccc003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.455+0000 7facddd9b640 1 --2- 172.21.15.19:0/2943273238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faccc0039c0 0x7faccc003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.455+0000 7facddd9b640 10 monclient: _finish_hunting 0 2023-10-06T13:53:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.455+0000 7facddd9b640 1 monclient: found mon.c 2023-10-06T13:53:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.455+0000 7facddd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.455+0000 7facddd9b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.455+0000 7facddd9b640 1 -- 172.21.15.19:0/2943273238 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc003520 con 0x7faccc0047c0 2023-10-06T13:53:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.455+0000 7facddd9b640 10 monclient: _finish_auth 0 2023-10-06T13:53:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.455+0000 7facddd9b640 10 monclient: _check_auth_tickets 2023-10-06T13:53:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.455+0000 7facddd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.455+0000 7facded9d640 1 -- 172.21.15.19:0/2943273238 reap_dead start 2023-10-06T13:53:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.457+0000 7facddd9b640 1 --2- 172.21.15.19:0/2943273238 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faccc0047c0 0x7faccc004c30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7facc8009860 tx=0x7faccc0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f997a682c034af6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.460+0000 7facdd59a640 1 -- 172.21.15.19:0/2943273238 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8032600 con 0x7faccc0047c0 2023-10-06T13:53:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.460+0000 7facdd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.460+0000 7facdd59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.460+0000 7facdd59a640 10 monclient: dump: 2023-10-06T13:53:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:18.464 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.460+0000 7facdd59a640 1 -- 172.21.15.19:0/2943273238 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc802e050 con 0x7faccc0047c0 2023-10-06T13:53:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.460+0000 7facdd59a640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.460+0000 7facdd59a640 1 -- 172.21.15.19:0/2943273238 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8032a00 con 0x7faccc0047c0 2023-10-06T13:53:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.460+0000 7face4a8b640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.460+0000 7face4a8b640 4 set_mon_vals no callback set 2023-10-06T13:53:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.460+0000 7facdd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.460+0000 7facdd59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:18.465 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.460+0000 7facdd59a640 10 monclient: dump: 2023-10-06T13:53:18.465 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:18.465 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:18.465 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:18.465 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:18.465 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:18.465 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:18.465 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:18.465 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:18.465 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:18.465 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:18.465 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.461+0000 7face4a8b640 10 monclient: shutdown 2023-10-06T13:53:18.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.461+0000 7face4a8b640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:18.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.461+0000 7face4a8b640 1 -- 172.21.15.19:0/2943273238 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faccc0047c0 msgr2=0x7faccc004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:18.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.461+0000 7face4a8b640 1 --2- 172.21.15.19:0/2943273238 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faccc0047c0 0x7faccc004c30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7facc8009860 tx=0x7faccc0057f0 comp rx=0 tx=0).stop 2023-10-06T13:53:18.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.461+0000 7face4a8b640 1 -- 172.21.15.19:0/2943273238 shutdown_connections 2023-10-06T13:53:18.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.461+0000 7face4a8b640 1 --2- 172.21.15.19:0/2943273238 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faccc0047c0 0x7faccc004c30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:18.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.461+0000 7face4a8b640 1 -- 172.21.15.19:0/2943273238 >> 172.21.15.19:0/2943273238 conn(0x7face00665b0 msgr2=0x7face006ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:18.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.461+0000 7face4a8b640 1 -- 172.21.15.19:0/2943273238 shutdown_connections 2023-10-06T13:53:18.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.461+0000 7face4a8b640 1 -- 172.21.15.19:0/2943273238 wait complete. 2023-10-06T13:53:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.462+0000 7face4a8b640 10 monclient: build_initial_monmap 2023-10-06T13:53:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.462+0000 7face4a8b640 10 monclient: monmap: 2023-10-06T13:53:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:18.469 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.462+0000 7face4a8b640 1 librados: starting msgr at 2023-10-06T13:53:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.462+0000 7face4a8b640 1 librados: starting objecter 2023-10-06T13:53:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.462+0000 7face4a8b640 1 Processor -- start 2023-10-06T13:53:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.462+0000 7face4a8b640 1 -- start start 2023-10-06T13:53:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.462+0000 7face4a8b640 1 librados: setting wanted keys 2023-10-06T13:53:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.462+0000 7face4a8b640 1 librados: calling monclient init 2023-10-06T13:53:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.462+0000 7face4a8b640 10 monclient: init 2023-10-06T13:53:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 10 monclient: _add_conns ranks=[2,1,0] 2023-10-06T13:53:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faccc0047c0 0x7face0141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 10 monclient(hunting): picked mon.noname-c con 0x7faccc0047c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face00a8900 0x7face01d5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 10 monclient(hunting): picked mon.noname-b con 0x7face00a8900 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face01d5ff0 0x7face01d8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 10 monclient(hunting): picked mon.noname-a con 0x7face01d5ff0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face0140690 con 0x7face01d5ff0 2023-10-06T13:53:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face01d55b0 con 0x7face00a8900 2023-10-06T13:53:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face00a8d30 con 0x7faccc0047c0 2023-10-06T13:53:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7face4a8b640 10 monclient(hunting): authenticate will time out at 1326.649674s 2023-10-06T13:53:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7facde59c640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faccc0047c0 0x7face0141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.463+0000 7facded9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face01d5ff0 0x7face01d8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facded9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face01d5ff0 0x7face01d8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38752/0 (socket says 172.21.15.19:38752) 2023-10-06T13:53:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facded9d640 1 -- 172.21.15.19:0/2758630503 learned_addr learned my addr 172.21.15.19:0/2758630503 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facded9d640 10 monclient(hunting): get_auth_request con 0x7face01d5ff0 auth_method 0 2023-10-06T13:53:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facded9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facded9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facded9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facde59c640 10 monclient(hunting): get_auth_request con 0x7faccc0047c0 auth_method 0 2023-10-06T13:53:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facde59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facde59c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facde59c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facde59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facde59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facde59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facded9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facded9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.464+0000 7facded9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:18.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.467+0000 7facded9d640 10 monclient(hunting): handle_auth_done global_id 4257 payload 824 2023-10-06T13:53:18.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.467+0000 7facded9d640 1 -- 172.21.15.19:0/2758630503 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faccc0047c0 msgr2=0x7face0141460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:18.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.467+0000 7facded9d640 1 --2- 172.21.15.19:0/2758630503 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faccc0047c0 0x7face0141460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:18.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.467+0000 7facded9d640 1 -- 172.21.15.19:0/2758630503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face00a8900 msgr2=0x7face01d5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:18.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.467+0000 7facded9d640 1 --2- 172.21.15.19:0/2758630503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face00a8900 0x7face01d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.467+0000 7facded9d640 10 monclient: _finish_hunting 0 2023-10-06T13:53:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.467+0000 7facded9d640 1 monclient: found mon.noname-a 2023-10-06T13:53:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.467+0000 7facded9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.467+0000 7facded9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.467+0000 7facded9d640 1 -- 172.21.15.19:0/2758630503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face013eb70 con 0x7face01d5ff0 2023-10-06T13:53:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.469+0000 7facded9d640 1 --2- 172.21.15.19:0/2758630503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face01d5ff0 0x7face01d8b30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7facd0001370 tx=0x7facd000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.469+0000 7facd6ffd640 10 client.?.objecter ms_handle_connect 0x7face01d5ff0 2023-10-06T13:53:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.469+0000 7facd6ffd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7facd6ffd640 1 -- 172.21.15.19:0/2758630503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd0004d60 con 0x7face01d5ff0 2023-10-06T13:53:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7facd6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7facd6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7facd6ffd640 10 monclient: dump: 2023-10-06T13:53:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:18.478 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:18.478 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:18.478 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:18.478 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:18.478 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:18.478 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:18.478 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:18.478 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:18.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7facd6ffd640 10 monclient: _finish_auth 0 2023-10-06T13:53:18.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7facd6ffd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:18.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7facd6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:18.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7facd6ffd640 1 -- 172.21.15.19:0/2758630503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd0007110 con 0x7face01d5ff0 2023-10-06T13:53:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7face4a8b640 5 monclient: authenticate success, global_id 4257 2023-10-06T13:53:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7face4a8b640 10 monclient: _renew_subs 2023-10-06T13:53:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7face4a8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7face4a8b640 1 -- 172.21.15.19:0/2758630503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face01d6460 con 0x7face01d5ff0 2023-10-06T13:53:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7facd6ffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7facd6ffd640 1 -- 172.21.15.19:0/2758630503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd00074c0 con 0x7face01d5ff0 2023-10-06T13:53:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7facd6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7facd6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7facd6ffd640 10 monclient: dump: 2023-10-06T13:53:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:18.480 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7facdcd99640 4 set_mon_vals no callback set 2023-10-06T13:53:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7face4a8b640 10 client.4257.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7face4a8b640 10 monclient: _renew_subs 2023-10-06T13:53:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7face4a8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7face4a8b640 1 -- 172.21.15.19:0/2758630503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face0141c00 con 0x7face01d5ff0 2023-10-06T13:53:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.474+0000 7face4a8b640 1 librados: init done 2023-10-06T13:53:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.475+0000 7face4a8b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.475+0000 7face4a8b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.475+0000 7face4a8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.475+0000 7face4a8b640 1 -- 172.21.15.19:0/2758630503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face004e000 con 0x7face01d5ff0 2023-10-06T13:53:18.498 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.496+0000 7facd6ffd640 1 -- 172.21.15.19:0/2758630503 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7facd0007620 con 0x7face01d5ff0 2023-10-06T13:53:18.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.497+0000 7facd6ffd640 1 --2- 172.21.15.19:0/2758630503 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7facc006c1c0 0x7facc006e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:18.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.497+0000 7facde59c640 1 --2- 172.21.15.19:0/2758630503 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7facc006c1c0 0x7facc006e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:18.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.497+0000 7facde59c640 10 monclient: get_auth_request con 0x7facc006c1c0 auth_method 0 2023-10-06T13:53:18.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.497+0000 7facd6ffd640 10 client.4257.objecter ms_handle_connect 0x7facc006c1c0 2023-10-06T13:53:18.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.497+0000 7facde59c640 1 --2- 172.21.15.19:0/2758630503 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7facc006c1c0 0x7facc006e670 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7facd8007fd0 tx=0x7facd8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.501+0000 7facd6ffd640 1 -- 172.21.15.19:0/2758630503 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7facd0057fe0 con 0x7face01d5ff0 2023-10-06T13:53:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.501+0000 7facd6ffd640 10 client.4257.objecter ms_dispatch 0x7face0000b90 osd_map(22..22 src has 1..22) v4 2023-10-06T13:53:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.501+0000 7facd6ffd640 3 client.4257.objecter handle_osd_map got epochs [22,22] > 0 2023-10-06T13:53:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.501+0000 7facd6ffd640 3 client.4257.objecter handle_osd_map decoding full epoch 22 2023-10-06T13:53:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.501+0000 7facd6ffd640 20 client.4257.objecter dump_active .. 0 homeless 2023-10-06T13:53:18.764 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.762+0000 7facd6ffd640 1 -- 172.21.15.19:0/2758630503 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7facd0053c20 con 0x7face01d5ff0 2023-10-06T13:53:18.764 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.762+0000 7facd6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:18.764 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.762+0000 7facd6ffd640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.948+0000 7face4a8b640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:53:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.948+0000 7face4a8b640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:53:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.948+0000 7face4a8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:18.948+0000 7face4a8b640 1 -- 172.21.15.19:0/2758630503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7face006ad30 con 0x7face01d5ff0 2023-10-06T13:53:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.070+0000 7facd6ffd640 1 -- 172.21.15.19:0/2758630503 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+14146 (secure 0 0 0) 0x7facd0057240 con 0x7face01d5ff0 2023-10-06T13:53:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.070+0000 7facd6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T13:53:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.070+0000 7facd6ffd640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:19.072 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:53:19.073 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":23,"fsid":"c5a90f7c-81ec-43a3-ab47-af6ce5729ec5","created":"2023-10-06T13:52:19.516184+0000","modified":"2023-10-06T13:53:18.746662+0000","last_up_change":"2023-10-06T13:53:08.156870+0000","last_in_change":"2023-10-06T13:52:24.916896+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T13:53:07.784836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T13:53:14.364162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6cb75c5-86fb-48b9-895c-fb1d72bd4746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":86195},{"type":"v1","addr":"172.21.15.19:6818","nonce":86195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6821","nonce":86195},{"type":"v1","addr":"172.21.15.19:6823","nonce":86195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":86195},{"type":"v1","addr":"172.21.15.19:6831","nonce":86195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":86195},{"type":"v1","addr":"172.21.15.19:6827","nonce":86195}]},"public_addr":"172.21.15.19:6818/86195","cluster_addr":"172.21.15.19:6823/86195","heartbeat_back_addr":"172.21.15.19:6831/86195","heartbeat_front_addr":"172.21.15.19:6827/86195","state":["exists","up"]},{"osd":1,"uuid":"ba828cb4-a4f6-4fea-a02e-24d9cf597ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":86193},{"type":"v1","addr":"172.21.15.19:6819","nonce":86193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":86193},{"type":"v1","addr":"172.21.15.19:6822","nonce":86193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":86193},{"type":"v1","addr":"172.21.15.19:6830","nonce":86193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":86193},{"type":"v1","addr":"172.21.15.19:6826","nonce":86193}]},"public_addr":"172.21.15.19:6819/86193","cluster_addr":"172.21.15.19:6822/86193","heartbeat_back_addr":"172.21.15.19:6830/86193","heartbeat_front_addr":"172.21.15.19:6826/86193","state":["exists","up"]},{"osd":2,"uuid":"06bcfab5-d2f1-4447-ba44-143398707ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86194},{"type":"v1","addr":"172.21.15.19:6809","nonce":86194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":86194},{"type":"v1","addr":"172.21.15.19:6811","nonce":86194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":86194},{"type":"v1","addr":"172.21.15.19:6815","nonce":86194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":86194},{"type":"v1","addr":"172.21.15.19:6813","nonce":86194}]},"public_addr":"172.21.15.19:6809/86194","cluster_addr":"172.21.15.19:6811/86194","heartbeat_back_addr":"172.21.15.19:6815/86194","heartbeat_front_addr":"172.21.15.19:6813/86194","state":["exists","up"]},{"osd":3,"uuid":"7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":86196},{"type":"v1","addr":"172.21.15.19:6801","nonce":86196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86196},{"type":"v1","addr":"172.21.15.19:6803","nonce":86196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86196},{"type":"v1","addr":"172.21.15.19:6807","nonce":86196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86196},{"type":"v1","addr":"172.21.15.19:6805","nonce":86196}]},"public_addr":"172.21.15.19:6801/86196","cluster_addr":"172.21.15.19:6803/86196","heartbeat_back_addr":"172.21.15.19:6807/86196","heartbeat_front_addr":"172.21.15.19:6805/86196","state":["exists","up"]},{"osd":4,"uuid":"ad31d4df-e78c-4e32-bfd4-3fe6e70dd428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":86123},{"type":"v1","addr":"172.21.15.83:6827","nonce":86123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":86123},{"type":"v1","addr":"172.21.15.83:6829","nonce":86123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":86123},{"type":"v1","addr":"172.21.15.83:6833","nonce":86123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":86123},{"type":"v1","addr":"172.21.15.83:6831","nonce":86123}]},"public_addr":"172.21.15.83:6827/86123","cluster_addr":"172.21.15.83:6829/86123","heartbeat_back_addr":"172.21.15.83:6833/86123","heartbeat_front_addr":"172.21.15.83:6831/86123","state":["exists","up"]},{"osd":5,"uuid":"931b23f8-80c3-4595-8931-2568be479cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":86126},{"type":"v1","addr":"172.21.15.83:6819","nonce":86126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":86126},{"type":"v1","addr":"172.21.15.83:6821","nonce":86126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":86126},{"type":"v1","addr":"172.21.15.83:6825","nonce":86126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":86126},{"type":"v1","addr":"172.21.15.83:6823","nonce":86126}]},"public_addr":"172.21.15.83:6819/86126","cluster_addr":"172.21.15.83:6821/86126","heartbeat_back_addr":"172.21.15.83:6825/86126","heartbeat_front_addr":"172.21.15.83:6823/86126","state":["exists","up"]},{"osd":6,"uuid":"70dd2f95-93eb-448a-a65d-3a09bdea85a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":86124},{"type":"v1","addr":"172.21.15.83:6803","nonce":86124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":86124},{"type":"v1","addr":"172.21.15.83:6805","nonce":86124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":86124},{"type":"v1","addr":"172.21.15.83:6809","nonce":86124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":86124},{"type":"v1","addr":"172.21.15.83:6807","nonce":86124}]},"public_addr":"172.21.15.83:6803/86124","cluster_addr":"172.21.15.83:6805/86124","heartbeat_back_addr":"172.21.15.83:6809/86124","heartbeat_front_addr":"172.21.15.83:6807/86124","state":["exists","up"]},{"osd":7,"uuid":"9797eb84-1b2c-4280-ab13-bdb7b2007dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":86125},{"type":"v1","addr":"172.21.15.83:6811","nonce":86125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":86125},{"type":"v1","addr":"172.21.15.83:6813","nonce":86125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":86125},{"type":"v1","addr":"172.21.15.83:6817","nonce":86125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":86125},{"type":"v1","addr":"172.21.15.83:6815","nonce":86125}]},"public_addr":"172.21.15.83:6811/86125","cluster_addr":"172.21.15.83:6813/86125","heartbeat_back_addr":"172.21.15.83:6817/86125","heartbeat_front_addr":"172.21.15.83:6815/86125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:04.886714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:05.153337+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:04.794593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:04.049373+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:03.567476+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:02.921086+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:02.431856+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:02.863561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T13:53:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 10 librados: watch_flush enter 2023-10-06T13:53:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 10 librados: watch_flush exit 2023-10-06T13:53:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 20 client.4257.objecter shutdown clearing up homeless session... 2023-10-06T13:53:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 10 client.4257.objecter successfully canceled tick 2023-10-06T13:53:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 1 -- 172.21.15.19:0/2758630503 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7facc006c1c0 msgr2=0x7facc006e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 1 --2- 172.21.15.19:0/2758630503 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7facc006c1c0 0x7facc006e670 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7facd8007fd0 tx=0x7facd8009040 comp rx=0 tx=0).stop 2023-10-06T13:53:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 10 monclient: shutdown 2023-10-06T13:53:19.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 1 -- 172.21.15.19:0/2758630503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face01d5ff0 msgr2=0x7face01d8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 1 --2- 172.21.15.19:0/2758630503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face01d5ff0 0x7face01d8b30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7facd0001370 tx=0x7facd000c900 comp rx=0 tx=0).stop 2023-10-06T13:53:19.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 1 -- 172.21.15.19:0/2758630503 shutdown_connections 2023-10-06T13:53:19.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 1 --2- 172.21.15.19:0/2758630503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face01d5ff0 0x7face01d8b30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 1 --2- 172.21.15.19:0/2758630503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face00a8900 0x7face01d5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 1 --2- 172.21.15.19:0/2758630503 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7facc006c1c0 0x7facc006e670 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 1 --2- 172.21.15.19:0/2758630503 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faccc0047c0 0x7face0141460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 1 -- 172.21.15.19:0/2758630503 >> 172.21.15.19:0/2758630503 conn(0x7face00665b0 msgr2=0x7face00b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 1 -- 172.21.15.19:0/2758630503 shutdown_connections 2023-10-06T13:53:19.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 1 -- 172.21.15.19:0/2758630503 wait complete. 2023-10-06T13:53:19.077 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.072+0000 7face4a8b640 1 librados: shutdown 2023-10-06T13:53:19.085 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T13:53:19.085 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T13:53:19.086 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T13:53:19.086 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T13:53:19.086 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T13:53:19.087 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T13:53:19.087 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T13:53:19.088 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T13:53:19.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.271+0000 7f9e5e3f5640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.274 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.271+0000 7f9e5e3f5640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.274 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.271+0000 7f9e5e3f5640 10 monclient: monmap: 2023-10-06T13:53:19.274 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.274 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.274 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.272874+0000 2023-10-06T13:53:19.274 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.272874+0000 2023-10-06T13:53:19.274 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.274 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.274 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.274 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.275 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.275 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.275 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.275 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.275 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.270+0000 7ffb7d280640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.275 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.270+0000 7ffb7d280640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.275 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.270+0000 7ffb7d280640 10 monclient: monmap: 2023-10-06T13:53:19.275 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.271953+0000 2023-10-06T13:53:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.271953+0000 2023-10-06T13:53:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.276 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.273+0000 7f9e5e3f5640 1 Processor -- start 2023-10-06T13:53:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.273+0000 7f9e5e3f5640 1 -- start start 2023-10-06T13:53:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.273+0000 7f9e5e3f5640 10 monclient: init 2023-10-06T13:53:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.273+0000 7f9e5e3f5640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.274+0000 7f9e5e3f5640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2023-10-06T13:53:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.274+0000 7f9e5e3f5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9e50031580 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.274+0000 7f9e5e3f5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9e500c8d50 0x7f9e500cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.274+0000 7f9e5e3f5640 10 monclient(hunting): picked mon.noname-c con 0x7f9e500c8d50 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.274+0000 7f9e5e3f5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9e50030c80 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.274+0000 7f9e5e3f5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e50031950 con 0x7f9e50030c80 2023-10-06T13:53:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.274+0000 7f9e5e3f5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e500305d0 con 0x7f9e50031580 2023-10-06T13:53:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.274+0000 7f9e5e3f5640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.274+0000 7ffb7d280640 1 Processor -- start 2023-10-06T13:53:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.274+0000 7f9e5e3f5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e50030ab0 con 0x7f9e500c8d50 2023-10-06T13:53:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.274+0000 7f9e5e3f5640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.274+0000 7f9e5e3f5640 10 monclient(hunting): authenticate will time out at 1327.460466s 2023-10-06T13:53:19.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7fa062884640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7fa062884640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7fa062884640 10 monclient: monmap: 2023-10-06T13:53:19.280 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.280 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.280 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.276295+0000 2023-10-06T13:53:19.280 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.276295+0000 2023-10-06T13:53:19.280 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.281 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7f9e577fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9e500c8d50 0x7f9e500cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.274+0000 7ffb7d280640 1 -- start start 2023-10-06T13:53:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.274+0000 7ffb7d280640 10 monclient: init 2023-10-06T13:53:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7ffb7d280640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7ffb7d280640 10 monclient: _add_conns ranks=[2,4,0,3,1] 2023-10-06T13:53:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7ffb7d280640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffb7806da00 0x7ffb78140420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7ffb7d280640 10 monclient(hunting): picked mon.noname-b con 0x7ffb7806da00 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7ffb7d280640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb780aca90 0x7ffb78140960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7ffb7d280640 10 monclient(hunting): picked mon.noname-c con 0x7ffb780aca90 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7ffb7d280640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffb780714c0 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7ffb7d280640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb7806ddd0 con 0x7ffb780714c0 2023-10-06T13:53:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7ffb7d280640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7ffb7d280640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb78140ea0 con 0x7ffb780aca90 2023-10-06T13:53:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7ffb7d280640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7ffb7d280640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7806eb80 con 0x7ffb7806da00 2023-10-06T13:53:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7ffb7d280640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.275+0000 7ffb7d280640 10 monclient(hunting): authenticate will time out at 1327.462014s 2023-10-06T13:53:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.276+0000 7f9e5c992640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9e50030c80 0x7f9e500cbb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57870/0 (socket says 172.21.15.19:57870) 2023-10-06T13:53:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.276+0000 7f9e5c992640 1 -- 172.21.15.19:0/3600435878 learned_addr learned my addr 172.21.15.19:0/3600435878 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.276+0000 7f9e577fe640 10 monclient(hunting): get_auth_request con 0x7f9e500c8d50 auth_method 0 2023-10-06T13:53:19.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.276+0000 7f9e577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.276+0000 7f9e577fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.276+0000 7f9e577fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.276+0000 7fa062884640 1 Processor -- start 2023-10-06T13:53:19.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.276+0000 7fa062884640 1 -- start start 2023-10-06T13:53:19.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.276+0000 7fa062884640 10 monclient: init 2023-10-06T13:53:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7f9e577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7f9e577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7f9e577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7ffb777fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb780aca90 0x7ffb78140960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7ffb77fff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffb7806da00 0x7ffb78140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7ffb77fff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffb7806da00 0x7ffb78140420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:41242/0 (socket says 172.21.15.19:41242) 2023-10-06T13:53:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7ffb77fff640 1 -- 172.21.15.19:0/823673005 learned_addr learned my addr 172.21.15.19:0/823673005 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7ffb77fff640 10 monclient(hunting): get_auth_request con 0x7ffb7806da00 auth_method 0 2023-10-06T13:53:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7ffb77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7ffb77fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7ffb77fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7fa062884640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7fa062884640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2023-10-06T13:53:19.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7fa062884640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa05c069e70 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:19.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7fa062884640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa05c13e2b0 0x7fa05c13e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7fa062884640 10 monclient(hunting): picked mon.noname-c con 0x7fa05c13e2b0 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:19.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7fa062884640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa05c149790 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:19.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7fa062884640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa05c06a240 con 0x7fa05c069e70 2023-10-06T13:53:19.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7fa062884640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa05c06db70 con 0x7fa05c149790 2023-10-06T13:53:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7fa062884640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7fa062884640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa05c06bb40 con 0x7fa05c13e2b0 2023-10-06T13:53:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7fa062884640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.277+0000 7fa062884640 10 monclient(hunting): authenticate will time out at 1327.464330s 2023-10-06T13:53:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.278+0000 7ffb77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.278+0000 7ffb77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.278+0000 7ffb77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.279+0000 7f9e577fe640 10 monclient(hunting): handle_auth_done global_id 4300 payload 274 2023-10-06T13:53:19.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.279+0000 7f9e577fe640 1 -- 172.21.15.19:0/3600435878 >> v1:172.21.15.83:6789/0 conn(0x7f9e50031580 legacy=0x7f9e500c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.279+0000 7f9e577fe640 1 -- 172.21.15.19:0/3600435878 >> v1:172.21.15.19:6789/0 conn(0x7f9e50030c80 legacy=0x7f9e500cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.279+0000 7f9e577fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.279+0000 7f9e577fe640 1 monclient: found mon.noname-c 2023-10-06T13:53:19.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.279+0000 7f9e577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.290 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.279+0000 7f9e577fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.290 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.279+0000 7f9e577fe640 1 -- 172.21.15.19:0/3600435878 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e50032540 con 0x7f9e500c8d50 2023-10-06T13:53:19.290 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.280+0000 7ffb777fe640 10 monclient(hunting): get_auth_request con 0x7ffb780aca90 auth_method 0 2023-10-06T13:53:19.290 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.280+0000 7ffb777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.290 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.280+0000 7ffb777fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.290 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.280+0000 7ffb777fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.279+0000 7fbe0c753640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.279+0000 7fbe0c753640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.279+0000 7fbe0c753640 10 monclient: monmap: 2023-10-06T13:53:19.291 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.291 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.291 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.280746+0000 2023-10-06T13:53:19.291 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.280746+0000 2023-10-06T13:53:19.291 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.291 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.291 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.292 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.280+0000 7fbe0c753640 1 Processor -- start 2023-10-06T13:53:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.280+0000 7fbe0c753640 1 -- start start 2023-10-06T13:53:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.280+0000 7fbe0c753640 10 monclient: init 2023-10-06T13:53:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.281+0000 7ffb77fff640 10 monclient(hunting): handle_auth_done global_id 4235 payload 274 2023-10-06T13:53:19.293 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.281+0000 7ffb77fff640 1 -- 172.21.15.19:0/823673005 >> v2:172.21.15.83:3300/0 conn(0x7ffb780aca90 msgr2=0x7ffb78140960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.293 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.281+0000 7ffb77fff640 1 --2- 172.21.15.19:0/823673005 >> v2:172.21.15.83:3300/0 conn(0x7ffb780aca90 0x7ffb78140960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.293 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.281+0000 7ffb77fff640 1 -- 172.21.15.19:0/823673005 >> v1:172.21.15.19:6789/0 conn(0x7ffb780714c0 legacy=0x7ffb78139bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.293 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.281+0000 7ffb77fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.293 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.281+0000 7ffb77fff640 1 monclient: found mon.noname-b 2023-10-06T13:53:19.293 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.281+0000 7ffb77fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.293 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.281+0000 7ffb77fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.293 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.281+0000 7ffb77fff640 1 -- 172.21.15.19:0/823673005 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7813ff00 con 0x7ffb7806da00 2023-10-06T13:53:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.282+0000 7fbe0c753640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.282+0000 7fbe0c753640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2023-10-06T13:53:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.282+0000 7fbe0c753640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbe040afd00 0x7fbe040a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.282+0000 7fbe0c753640 10 monclient(hunting): picked mon.noname-c con 0x7fbe040afd00 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.282+0000 7fbe0c753640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbe0413da90 0x7fbe0413dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.282+0000 7fbe0c753640 10 monclient(hunting): picked mon.noname-a con 0x7fbe0413da90 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.282+0000 7fbe0c753640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbe041440a0 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.282+0000 7fbe0c753640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe040b00d0 con 0x7fbe041440a0 2023-10-06T13:53:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.282+0000 7fbe0c753640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.282+0000 7fbe0c753640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe040b1ac0 con 0x7fbe0413da90 2023-10-06T13:53:19.295 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.282+0000 7fbe0c753640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.295 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.282+0000 7fbe0c753640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe040a95d0 con 0x7fbe040afd00 2023-10-06T13:53:19.295 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.282+0000 7fbe0c753640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.295 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.282+0000 7fbe0c753640 10 monclient(hunting): authenticate will time out at 1327.468469s 2023-10-06T13:53:19.295 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.283+0000 7ffb77fff640 1 --2- 172.21.15.19:0/823673005 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffb7806da00 0x7ffb78140420 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00c970 tx=0x7ffb6c02dd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23d5b8483914c4de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.295 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.284+0000 7fbe09cee640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbe0413da90 0x7fbe0413dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.284+0000 7f9e577fe640 1 --2- 172.21.15.19:0/3600435878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e500c8d50 0x7f9e500cb610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9e480097a0 tx=0x7f9e4802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31758cddf3110834 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.284+0000 7fbe09cee640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbe0413da90 0x7fbe0413dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38756/0 (socket says 172.21.15.19:38756) 2023-10-06T13:53:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.285+0000 7fbe09cee640 1 -- 172.21.15.19:0/2241411585 learned_addr learned my addr 172.21.15.19:0/2241411585 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.285+0000 7fbe09cee640 10 monclient(hunting): get_auth_request con 0x7fbe0413da90 auth_method 0 2023-10-06T13:53:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.285+0000 7fa799aaf640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.285+0000 7fa799aaf640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.285+0000 7fa799aaf640 10 monclient: monmap: 2023-10-06T13:53:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.286167+0000 2023-10-06T13:53:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.286167+0000 2023-10-06T13:53:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.298 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.298 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.298 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.298 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.298 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.298 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.298 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.298 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.285+0000 7fbe09cee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.298 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.285+0000 7fbe09cee640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.285+0000 7fbe09cee640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.285+0000 7ffaa567a640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.285+0000 7ffaa567a640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.285+0000 7ffaa567a640 10 monclient: monmap: 2023-10-06T13:53:19.299 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.299 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.300 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.286333+0000 2023-10-06T13:53:19.300 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.286333+0000 2023-10-06T13:53:19.300 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.300 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.300 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.300 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.300 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.300 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.300 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.300 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7fbe09cee640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7fbe09cee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7fbe09cee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7ffaa567a640 1 Processor -- start 2023-10-06T13:53:19.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7fa799aaf640 1 Processor -- start 2023-10-06T13:53:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7fa799aaf640 1 -- start start 2023-10-06T13:53:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7fa799aaf640 10 monclient: init 2023-10-06T13:53:19.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7ffb76ffd640 1 -- 172.21.15.19:0/823673005 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c02eb30 con 0x7ffb7806da00 2023-10-06T13:53:19.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7ffb76ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7ffb76ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:19.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7ffb76ffd640 10 monclient: dump: 2023-10-06T13:53:19.303 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.303 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.303 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.303 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.303 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.303 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.303 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.304 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.304 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.304 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7ffb76ffd640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7ffb76ffd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7ffb76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7ffb7d280640 5 monclient: authenticate success, global_id 4235 2023-10-06T13:53:19.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7ffb7d280640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7ffaa567a640 1 -- start start 2023-10-06T13:53:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.286+0000 7ffaa567a640 10 monclient: init 2023-10-06T13:53:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7ffaa567a640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7ffaa567a640 10 monclient: _add_conns ranks=[4,1,3,0,2] 2023-10-06T13:53:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7ffaa567a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffaa00b3990 0x7ffaa00af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7ffaa567a640 10 monclient(hunting): picked mon.noname-c con 0x7ffaa00b3990 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7ffaa567a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ffaa00b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa799aaf640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa799aaf640 10 monclient: _add_conns ranks=[2,1,4,0,3] 2023-10-06T13:53:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa799aaf640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa7940b3990 0x7fa7940af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa799aaf640 10 monclient(hunting): picked mon.noname-b con 0x7fa7940b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa799aaf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa7940b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa799aaf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa7940b00c0 0x7fa7940b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7ffaa567a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffaa00b00c0 0x7ffaa00b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa799aaf640 10 monclient(hunting): picked mon.noname-c con 0x7fa7940b00c0 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:19.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa799aaf640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7940aebe0 con 0x7fa7940b3d60 2023-10-06T13:53:19.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa799aaf640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa799aaf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7940afd50 con 0x7fa7940b00c0 2023-10-06T13:53:19.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa799aaf640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa799aaf640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa794140dd0 con 0x7fa7940b3990 2023-10-06T13:53:19.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa799aaf640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa799aaf640 10 monclient(hunting): authenticate will time out at 1327.474008s 2023-10-06T13:53:19.308 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa060e21640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa05c149790 0x7fa05c14bb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:38280/0 (socket says 172.21.15.19:38280) 2023-10-06T13:53:19.308 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7fa060e21640 1 -- 172.21.15.19:0/2691752570 learned_addr learned my addr 172.21.15.19:0/2691752570 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:19.308 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7ffaa567a640 10 monclient(hunting): picked mon.noname-a con 0x7ffaa00b00c0 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:19.308 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7ffaa567a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa00aebe0 con 0x7ffaa00b3d60 2023-10-06T13:53:19.308 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7ffaa567a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.308 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7ffaa567a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa00b0a60 con 0x7ffaa00b00c0 2023-10-06T13:53:19.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7ffaa567a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7ffaa567a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa0140dd0 con 0x7ffaa00b3990 2023-10-06T13:53:19.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7ffaa567a640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.287+0000 7ffaa567a640 10 monclient(hunting): authenticate will time out at 1327.473621s 2023-10-06T13:53:19.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.288+0000 7fbe09cee640 10 monclient(hunting): handle_auth_done global_id 4260 payload 274 2023-10-06T13:53:19.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.288+0000 7fbe09cee640 1 -- 172.21.15.19:0/2241411585 >> v2:172.21.15.83:3300/0 conn(0x7fbe040afd00 msgr2=0x7fbe040a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.288+0000 7fbe09cee640 1 --2- 172.21.15.19:0/2241411585 >> v2:172.21.15.83:3300/0 conn(0x7fbe040afd00 0x7fbe040a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.288+0000 7fbe09cee640 1 -- 172.21.15.19:0/2241411585 >> v1:172.21.15.83:6789/0 conn(0x7fbe041440a0 legacy=0x7fbe04144500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.288+0000 7fbe09cee640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.288+0000 7fbe09cee640 1 monclient: found mon.noname-a 2023-10-06T13:53:19.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.288+0000 7fbe09cee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.288+0000 7fbe09cee640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.288+0000 7fbe09cee640 1 -- 172.21.15.19:0/2241411585 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe040a98b0 con 0x7fbe0413da90 2023-10-06T13:53:19.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.288+0000 7ff7d000b640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.311 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.288+0000 7ff7d000b640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.311 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.288+0000 7ff7d000b640 10 monclient: monmap: 2023-10-06T13:53:19.311 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.311 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.311 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.289963+0000 2023-10-06T13:53:19.311 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.289963+0000 2023-10-06T13:53:19.311 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.311 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.311 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.311 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.311 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.311 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.312 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.312 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.289+0000 7ffb76ffd640 1 -- 172.21.15.19:0/823673005 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c032070 con 0x7ffb7806da00 2023-10-06T13:53:19.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.289+0000 7ffb76ffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.289+0000 7ff7d000b640 1 Processor -- start 2023-10-06T13:53:19.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.289+0000 7ffb7d280640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.289+0000 7ffb7d280640 4 set_mon_vals no callback set 2023-10-06T13:53:19.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.289+0000 7ffb76ffd640 1 -- 172.21.15.19:0/823673005 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c038400 con 0x7ffb7806da00 2023-10-06T13:53:19.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.289+0000 7ffb76ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.289+0000 7ffb76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:19.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.289+0000 7ffb76ffd640 10 monclient: dump: 2023-10-06T13:53:19.313 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.313 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.313 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.314 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.314 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.314 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.314 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.314 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.314 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.314 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7fa7937fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa7940b3990 0x7fa7940af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7ff7d000b640 1 -- start start 2023-10-06T13:53:19.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7ff7d000b640 10 monclient: init 2023-10-06T13:53:19.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7fa7937fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa7940b3990 0x7fa7940af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:41252/0 (socket says 172.21.15.19:41252) 2023-10-06T13:53:19.316 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7fa7937fe640 1 -- 172.21.15.19:0/446819031 learned_addr learned my addr 172.21.15.19:0/446819031 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.316 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7fa7937fe640 10 monclient(hunting): get_auth_request con 0x7fa7940b3990 auth_method 0 2023-10-06T13:53:19.316 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7fa7937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.316 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7fa7937fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.316 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7ffb7d280640 10 monclient: shutdown 2023-10-06T13:53:19.316 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7fa7937fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.317 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7ffb7d280640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.317 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7ffb7d280640 1 -- 172.21.15.19:0/823673005 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffb7806da00 msgr2=0x7ffb78140420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.317 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7ffb7d280640 1 --2- 172.21.15.19:0/823673005 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffb7806da00 0x7ffb78140420 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00c970 tx=0x7ffb6c02dd80 comp rx=0 tx=0).stop 2023-10-06T13:53:19.317 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7ffb7d280640 1 -- 172.21.15.19:0/823673005 shutdown_connections 2023-10-06T13:53:19.317 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7ffb7d280640 1 --2- 172.21.15.19:0/823673005 >> v2:172.21.15.83:3300/0 conn(0x7ffb780aca90 0x7ffb78140960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.317 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7ffb7d280640 1 --2- 172.21.15.19:0/823673005 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffb7806da00 0x7ffb78140420 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7ffb7d280640 1 -- 172.21.15.19:0/823673005 >> 172.21.15.19:0/823673005 conn(0x7ffb7806af30 msgr2=0x7ffb780b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7ffb7d280640 1 -- 172.21.15.19:0/823673005 shutdown_connections 2023-10-06T13:53:19.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.290+0000 7ffb7d280640 1 -- 172.21.15.19:0/823673005 wait complete. 2023-10-06T13:53:19.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7fa7937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7fa7937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7fa7937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ff7d000b640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ff7d000b640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2023-10-06T13:53:19.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ff7d000b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff7c00072c0 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:19.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ff7d000b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff7c0037260 0x7ff7c0034b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7fa05b7fe640 1 --2- 172.21.15.19:0/2691752570 >> v2:172.21.15.83:3300/0 conn(0x7fa05c13e2b0 0x7fa05c13e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ff7d000b640 10 monclient(hunting): picked mon.noname-c con 0x7ff7c0037260 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:19.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ffb7d280640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ffb7d280640 10 monclient: monmap: 2023-10-06T13:53:19.320 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.320 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.320 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.320 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.320 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.321 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.321 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.321 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.321 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.321 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ff7d000b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff7c00c67c0 0x7ff7c00c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ff7d000b640 10 monclient(hunting): picked mon.noname-a con 0x7ff7c00c67c0 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:19.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ff7d000b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c0007690 con 0x7ff7c00072c0 2023-10-06T13:53:19.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ff7d000b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ff7d000b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c00c7140 con 0x7ff7c00c67c0 2023-10-06T13:53:19.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ff7d000b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ff7d000b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c0035920 con 0x7ff7c0037260 2023-10-06T13:53:19.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ff7d000b640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ff7d000b640 10 monclient(hunting): authenticate will time out at 1327.478154s 2023-10-06T13:53:19.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ffb7d280640 1 librados: starting msgr at 2023-10-06T13:53:19.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.291+0000 7ffb7d280640 1 librados: starting objecter 2023-10-06T13:53:19.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7ffb7d280640 1 Processor -- start 2023-10-06T13:53:19.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7ffa967fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ffaa00b3d60 0x7ffaa0141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:38300/0 (socket says 172.21.15.19:38300) 2023-10-06T13:53:19.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7ffa967fc640 1 -- 172.21.15.19:0/3711152477 learned_addr learned my addr 172.21.15.19:0/3711152477 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:19.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7fa05b7fe640 10 monclient(hunting): get_auth_request con 0x7fa05c13e2b0 auth_method 0 2023-10-06T13:53:19.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7fa05b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7fa05b7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7fa05b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7ffb7d280640 1 -- start start 2023-10-06T13:53:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7ffb7d280640 1 librados: setting wanted keys 2023-10-06T13:53:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7ffb7d280640 1 librados: calling monclient init 2023-10-06T13:53:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7ffb7d280640 10 monclient: init 2023-10-06T13:53:19.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e56ffd640 1 -- 172.21.15.19:0/3600435878 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4802f050 con 0x7f9e500c8d50 2023-10-06T13:53:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e56ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e56ffd640 10 monclient: dump: 2023-10-06T13:53:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.327 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e56ffd640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e56ffd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e5e3f5640 5 monclient: authenticate success, global_id 4300 2023-10-06T13:53:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e5e3f5640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e56ffd640 1 -- 172.21.15.19:0/3600435878 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e48031700 con 0x7f9e500c8d50 2023-10-06T13:53:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e56ffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e5e3f5640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e5e3f5640 4 set_mon_vals no callback set 2023-10-06T13:53:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e56ffd640 1 -- 172.21.15.19:0/3600435878 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48031a90 con 0x7f9e500c8d50 2023-10-06T13:53:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e56ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.292+0000 7f9e56ffd640 10 monclient: dump: 2023-10-06T13:53:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.329 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 10 monclient: _add_conns ranks=[2,0,1] 2023-10-06T13:53:19.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffb7806da00 0x7ffb78137850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 10 monclient(hunting): picked mon.noname-c con 0x7ffb7806da00 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb780714c0 0x7ffb7813ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 10 monclient(hunting): picked mon.noname-a con 0x7ffb780714c0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb780aca90 0x7ffb781d2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 10 monclient(hunting): picked mon.noname-b con 0x7ffb780aca90 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780a88a0 con 0x7ffb780714c0 2023-10-06T13:53:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb78138360 con 0x7ffb780aca90 2023-10-06T13:53:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb781d1ce0 con 0x7ffb7806da00 2023-10-06T13:53:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7ffb7d280640 10 monclient(hunting): authenticate will time out at 1327.479551s 2023-10-06T13:53:19.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7fa05b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7fa05b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7fa05b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7fbe09cee640 1 --2- 172.21.15.19:0/2241411585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe0413da90 0x7fbe0413dea0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbe040b0310 tx=0x7fbdfc02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d168b8191fc8bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7f9e5e3f5640 10 monclient: shutdown 2023-10-06T13:53:19.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7f9e5e3f5640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7f9e5e3f5640 1 -- 172.21.15.19:0/3600435878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e500c8d50 msgr2=0x7f9e500cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7f9e5e3f5640 1 --2- 172.21.15.19:0/3600435878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e500c8d50 0x7f9e500cb610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9e480097a0 tx=0x7f9e4802b7a0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7f9e5e3f5640 1 -- 172.21.15.19:0/3600435878 shutdown_connections 2023-10-06T13:53:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7f9e5e3f5640 1 --2- 172.21.15.19:0/3600435878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e500c8d50 0x7f9e500cb610 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7f9e5e3f5640 1 -- 172.21.15.19:0/3600435878 >> 172.21.15.19:0/3600435878 conn(0x7f9e50035e70 msgr2=0x7f9e50036270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7f9e5e3f5640 1 -- 172.21.15.19:0/3600435878 shutdown_connections 2023-10-06T13:53:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.293+0000 7f9e5e3f5640 1 -- 172.21.15.19:0/3600435878 wait complete. 2023-10-06T13:53:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.294+0000 7f9e5e3f5640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.294+0000 7f9e5e3f5640 10 monclient: monmap: 2023-10-06T13:53:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.334 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.294+0000 7f9e5e3f5640 1 librados: starting msgr at 2023-10-06T13:53:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.294+0000 7f9e5e3f5640 1 librados: starting objecter 2023-10-06T13:53:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.294+0000 7f9e5e3f5640 1 Processor -- start 2023-10-06T13:53:19.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa7937fe640 10 monclient(hunting): handle_auth_done global_id 4238 payload 274 2023-10-06T13:53:19.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa7937fe640 1 -- 172.21.15.19:0/446819031 >> v2:172.21.15.83:3300/0 conn(0x7fa7940b00c0 msgr2=0x7fa7940b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa7937fe640 1 --2- 172.21.15.19:0/446819031 >> v2:172.21.15.83:3300/0 conn(0x7fa7940b00c0 0x7fa7940b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa7937fe640 1 -- 172.21.15.19:0/446819031 >> v1:172.21.15.83:6789/0 conn(0x7fa7940b3d60 legacy=0x7fa794141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7f9e5e3f5640 1 -- start start 2023-10-06T13:53:19.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa7937fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa7937fe640 1 monclient: found mon.noname-b 2023-10-06T13:53:19.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa7937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa7937fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa7937fe640 1 -- 172.21.15.19:0/446819031 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa794141070 con 0x7fa7940b3990 2023-10-06T13:53:19.337 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7f9e5e3f5640 1 librados: setting wanted keys 2023-10-06T13:53:19.337 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7f9e5e3f5640 1 librados: calling monclient init 2023-10-06T13:53:19.337 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7f9e5e3f5640 10 monclient: init 2023-10-06T13:53:19.338 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa05b7fe640 10 monclient(hunting): handle_auth_done global_id 4309 payload 274 2023-10-06T13:53:19.338 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa05b7fe640 1 -- 172.21.15.19:0/2691752570 >> v1:172.21.15.83:6789/0 conn(0x7fa05c149790 legacy=0x7fa05c14bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.338 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa05b7fe640 1 -- 172.21.15.19:0/2691752570 >> v1:172.21.15.19:6789/0 conn(0x7fa05c069e70 legacy=0x7fa05c06d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.338 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa05b7fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.339 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa05b7fe640 1 monclient: found mon.noname-c 2023-10-06T13:53:19.339 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa05b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.339 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa05b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.339 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.295+0000 7fa05b7fe640 1 -- 172.21.15.19:0/2691752570 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c13ec40 con 0x7fa05c13e2b0 2023-10-06T13:53:19.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.296+0000 7f9e5e3f5640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.296+0000 7f9e5e3f5640 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T13:53:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.296+0000 7f9e5e3f5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e50030c80 0x7f9e500cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.296+0000 7f9e5e3f5640 10 monclient(hunting): picked mon.noname-b con 0x7f9e50030c80 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.296+0000 7f9e5e3f5640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9e50031580 0x7f9e500cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.296+0000 7f9e5e3f5640 10 monclient(hunting): picked mon.noname-c con 0x7f9e50031580 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.296+0000 7f9e5e3f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e500c8d50 0x7f9e501603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.296+0000 7f9e5e3f5640 10 monclient(hunting): picked mon.noname-a con 0x7f9e500c8d50 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.296+0000 7f9e5e3f5640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.297+0000 7f9e5e3f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e500bdc60 con 0x7f9e500c8d50 2023-10-06T13:53:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.297+0000 7f9e5e3f5640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.297+0000 7f9e5e3f5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e500cf1e0 con 0x7f9e50030c80 2023-10-06T13:53:19.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.297+0000 7f9e5e3f5640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.297+0000 7f9e5e3f5640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e500ce3d0 con 0x7f9e50031580 2023-10-06T13:53:19.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.297+0000 7f9e5e3f5640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.297+0000 7f9e5e3f5640 10 monclient(hunting): authenticate will time out at 1327.483814s 2023-10-06T13:53:19.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.298+0000 7fa7937fe640 1 --2- 172.21.15.19:0/446819031 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa7940b3990 0x7fa7940af810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa784009870 tx=0x7fa78402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=531c770b9726a7ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.298+0000 7ffa9f7fe640 1 --2- 172.21.15.19:0/3711152477 >> v2:172.21.15.19:3300/0 conn(0x7ffaa00b00c0 0x7ffaa00b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.299+0000 7ff7ce5a8640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff7c00c67c0 0x7ff7c00c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.299+0000 7ffa9f7fe640 10 monclient(hunting): get_auth_request con 0x7ffaa00b00c0 auth_method 0 2023-10-06T13:53:19.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.299+0000 7ffa9f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.299+0000 7ffa9f7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.299+0000 7ffa9f7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.299+0000 7ffa9f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.299+0000 7ffa9f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.299+0000 7ffa9f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.300+0000 7ff7ce5a8640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff7c00c67c0 0x7ff7c00c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38778/0 (socket says 172.21.15.19:38778) 2023-10-06T13:53:19.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.300+0000 7ff7ce5a8640 1 -- 172.21.15.19:0/2402175820 learned_addr learned my addr 172.21.15.19:0/2402175820 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.300+0000 7ff7ce5a8640 10 monclient(hunting): get_auth_request con 0x7ff7c00c67c0 auth_method 0 2023-10-06T13:53:19.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.300+0000 7ff7ce5a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.300+0000 7ff7ce5a8640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.300+0000 7ff7ce5a8640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.301+0000 7ff7ce5a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.301+0000 7ff7ce5a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.301+0000 7ff7ce5a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.346 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.302+0000 7ffb77fff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffb7806da00 0x7ffb78137850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.346 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ffb77fff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffb7806da00 0x7ffb78137850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:41254/0 (socket says 172.21.15.19:41254) 2023-10-06T13:53:19.346 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ffb77fff640 1 -- 172.21.15.19:0/1606106680 learned_addr learned my addr 172.21.15.19:0/1606106680 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.346 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ffb77fff640 10 monclient(hunting): get_auth_request con 0x7ffb7806da00 auth_method 0 2023-10-06T13:53:19.346 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ffb77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.346 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ffb77fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.346 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ffb77fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.347 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9e50031580 0x7f9e500cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.347 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ff7ce5a8640 10 monclient(hunting): handle_auth_done global_id 4269 payload 274 2023-10-06T13:53:19.347 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ff7ce5a8640 1 -- 172.21.15.19:0/2402175820 >> v2:172.21.15.83:3300/0 conn(0x7ff7c0037260 msgr2=0x7ff7c0034b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.347 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ff7ce5a8640 1 --2- 172.21.15.19:0/2402175820 >> v2:172.21.15.83:3300/0 conn(0x7ff7c0037260 0x7ff7c0034b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.347 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ff7ce5a8640 1 -- 172.21.15.19:0/2402175820 >> v1:172.21.15.19:6789/0 conn(0x7ff7c00072c0 legacy=0x7ff7c0035210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.347 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ff7ce5a8640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.347 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ff7ce5a8640 1 monclient: found mon.noname-a 2023-10-06T13:53:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ff7ce5a8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ff7ce5a8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.303+0000 7ff7ce5a8640 1 -- 172.21.15.19:0/2402175820 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c0037840 con 0x7ff7c00c67c0 2023-10-06T13:53:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.305+0000 7ffa9e7fc640 1 -- 172.21.15.19:0/3711152477 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038393194 0 0) 0x7ffaa00aebe0 con 0x7ffaa00b3d60 2023-10-06T13:53:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.305+0000 7ffa9e7fc640 10 monclient(hunting): _negotiate 2023-10-06T13:53:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.305+0000 7ffa9e7fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.305+0000 7ffa9e7fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.305+0000 7ffa9e7fc640 10 monclient(hunting): my global_id is 4312 2023-10-06T13:53:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.305+0000 7ffa9e7fc640 1 -- 172.21.15.19:0/3711152477 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa8c003520 con 0x7ffaa00b3d60 2023-10-06T13:53:19.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.305+0000 7ffa9effd640 1 --2- 172.21.15.19:0/3711152477 >> v2:172.21.15.83:3300/0 conn(0x7ffaa00b3990 0x7ffaa00af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.304+0000 7ffb77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.304+0000 7ffb77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.304+0000 7ffb77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.305+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9e50031580 0x7f9e500cebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:41262/0 (socket says 172.21.15.19:41262) 2023-10-06T13:53:19.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.305+0000 7f9e577fe640 1 -- 172.21.15.19:0/1227251426 learned_addr learned my addr 172.21.15.19:0/1227251426 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.305+0000 7f9e577fe640 10 monclient(hunting): get_auth_request con 0x7f9e50031580 auth_method 0 2023-10-06T13:53:19.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.305+0000 7f9e577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.305+0000 7f9e577fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.305+0000 7f9e577fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.308+0000 7ffa9f7fe640 10 monclient(hunting): handle_auth_done global_id 4266 payload 274 2023-10-06T13:53:19.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.308+0000 7ffa9f7fe640 1 -- 172.21.15.19:0/3711152477 >> v2:172.21.15.83:3300/0 conn(0x7ffaa00b3990 msgr2=0x7ffaa00af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.308+0000 7ffa9f7fe640 1 --2- 172.21.15.19:0/3711152477 >> v2:172.21.15.83:3300/0 conn(0x7ffaa00b3990 0x7ffaa00af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.308+0000 7ffa9f7fe640 1 -- 172.21.15.19:0/3711152477 >> v1:172.21.15.83:6789/0 conn(0x7ffaa00b3d60 legacy=0x7ffaa0141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffa9f7fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffa9f7fe640 1 monclient: found mon.noname-a 2023-10-06T13:53:19.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffa9f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffa9f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffa9f7fe640 1 -- 172.21.15.19:0/3711152477 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa0141070 con 0x7ffaa00b00c0 2023-10-06T13:53:19.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffb77fff640 10 monclient(hunting): handle_auth_done global_id 4241 payload 824 2023-10-06T13:53:19.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffb77fff640 1 -- 172.21.15.19:0/1606106680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb780aca90 msgr2=0x7ffb781d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffb77fff640 1 --2- 172.21.15.19:0/1606106680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb780aca90 0x7ffb781d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffb77fff640 1 -- 172.21.15.19:0/1606106680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb780714c0 msgr2=0x7ffb7813ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffb77fff640 1 --2- 172.21.15.19:0/1606106680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb780714c0 0x7ffb7813ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffb77fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffb77fff640 1 monclient: found mon.noname-c 2023-10-06T13:53:19.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffb77fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffb77fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.309+0000 7ffb77fff640 1 -- 172.21.15.19:0/1606106680 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c00c650 con 0x7ffb7806da00 2023-10-06T13:53:19.353 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.311+0000 7ff7ce5a8640 1 --2- 172.21.15.19:0/2402175820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7c00c67c0 0x7ff7c00c6c00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8072880 tx=0x7ff7c80a57a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0a160a5a3418b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.353 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.311+0000 7f9e577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.353 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.311+0000 7f9e577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.353 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.311+0000 7f9e577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.353 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.312+0000 7ffa9f7fe640 1 --2- 172.21.15.19:0/3711152477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00b00c0 0x7ffaa00b0520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0072330 tx=0x7ffa9802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba26b6cd9f27af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.354 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.316+0000 7fa05b7fe640 1 --2- 172.21.15.19:0/2691752570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa05c13e2b0 0x7fa05c13e700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa0500097a0 tx=0x7fa05002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2f8e4adb8fa3b22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.316+0000 7f9e577fe640 10 monclient(hunting): handle_auth_done global_id 4244 payload 824 2023-10-06T13:53:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.316+0000 7f9e577fe640 1 -- 172.21.15.19:0/1227251426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e50030c80 msgr2=0x7f9e500cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.316+0000 7f9e577fe640 1 --2- 172.21.15.19:0/1227251426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e50030c80 0x7f9e500cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.316+0000 7f9e577fe640 1 -- 172.21.15.19:0/1227251426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e500c8d50 msgr2=0x7f9e501603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.316+0000 7f9e577fe640 1 --2- 172.21.15.19:0/1227251426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e500c8d50 0x7f9e501603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.316+0000 7f9e577fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.316+0000 7f9e577fe640 1 monclient: found mon.noname-c 2023-10-06T13:53:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.316+0000 7f9e577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.316+0000 7f9e577fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.316+0000 7f9e577fe640 1 -- 172.21.15.19:0/1227251426 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e48009400 con 0x7f9e50031580 2023-10-06T13:53:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.317+0000 7fa792ffd640 1 -- 172.21.15.19:0/446819031 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7840317c0 con 0x7fa7940b3990 2023-10-06T13:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.317+0000 7fa792ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa792ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa792ffd640 10 monclient: dump: 2023-10-06T13:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.358 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa792ffd640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa792ffd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa792ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa792ffd640 1 -- 172.21.15.19:0/446819031 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa784031920 con 0x7fa7940b3990 2023-10-06T13:53:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa799aaf640 5 monclient: authenticate success, global_id 4238 2023-10-06T13:53:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa799aaf640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa792ffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa799aaf640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa792ffd640 1 -- 172.21.15.19:0/446819031 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784031cb0 con 0x7fa7940b3990 2023-10-06T13:53:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa799aaf640 4 set_mon_vals no callback set 2023-10-06T13:53:19.359 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa792ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.359 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa792ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:19.359 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa792ffd640 10 monclient: dump: 2023-10-06T13:53:19.359 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.359 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.359 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.359 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.359 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.359 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.359 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.359 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.359 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.360 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.360 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa799aaf640 10 monclient: shutdown 2023-10-06T13:53:19.360 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa799aaf640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.360 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa799aaf640 1 -- 172.21.15.19:0/446819031 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa7940b3990 msgr2=0x7fa7940af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.360 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa799aaf640 1 --2- 172.21.15.19:0/446819031 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa7940b3990 0x7fa7940af810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa784009870 tx=0x7fa78402b860 comp rx=0 tx=0).stop 2023-10-06T13:53:19.360 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa799aaf640 1 -- 172.21.15.19:0/446819031 shutdown_connections 2023-10-06T13:53:19.360 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa799aaf640 1 --2- 172.21.15.19:0/446819031 >> v2:172.21.15.83:3300/0 conn(0x7fa7940b00c0 0x7fa7940b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.360 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa799aaf640 1 --2- 172.21.15.19:0/446819031 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa7940b3990 0x7fa7940af810 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.360 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa799aaf640 1 -- 172.21.15.19:0/446819031 >> 172.21.15.19:0/446819031 conn(0x7fa7940b6e00 msgr2=0x7fa7940b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.360 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa799aaf640 1 -- 172.21.15.19:0/446819031 shutdown_connections 2023-10-06T13:53:19.360 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.318+0000 7fa799aaf640 1 -- 172.21.15.19:0/446819031 wait complete. 2023-10-06T13:53:19.361 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7ccda5640 1 -- 172.21.15.19:0/2402175820 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c80a9050 con 0x7ff7c00c67c0 2023-10-06T13:53:19.361 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7ccda5640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.361 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7ccda5640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7ccda5640 10 monclient: dump: 2023-10-06T13:53:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.362 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7ccda5640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7ccda5640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7ccda5640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7d000b640 5 monclient: authenticate success, global_id 4269 2023-10-06T13:53:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7ccda5640 1 -- 172.21.15.19:0/2402175820 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c80ab7c0 con 0x7ff7c00c67c0 2023-10-06T13:53:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7d000b640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7ccda5640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7d000b640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7d000b640 4 set_mon_vals no callback set 2023-10-06T13:53:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7ccda5640 1 -- 172.21.15.19:0/2402175820 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c80abc10 con 0x7ff7c00c67c0 2023-10-06T13:53:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7ccda5640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7ccda5640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7ff7ccda5640 10 monclient: dump: 2023-10-06T13:53:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.364 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.365 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7fa799aaf640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.365 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.319+0000 7fa799aaf640 10 monclient: monmap: 2023-10-06T13:53:19.365 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.365 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.365 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.365 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.365 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.365 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.366 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7fa799aaf640 1 librados: starting msgr at 2023-10-06T13:53:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7ff7d000b640 10 monclient: shutdown 2023-10-06T13:53:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7ff7d000b640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.367 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7ff7d000b640 1 -- 172.21.15.19:0/2402175820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7c00c67c0 msgr2=0x7ff7c00c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.367 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7ff7d000b640 1 --2- 172.21.15.19:0/2402175820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7c00c67c0 0x7ff7c00c6c00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8072880 tx=0x7ff7c80a57a0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.367 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7fa799aaf640 1 librados: starting objecter 2023-10-06T13:53:19.367 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7fa799aaf640 1 Processor -- start 2023-10-06T13:53:19.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7ff7d000b640 1 -- 172.21.15.19:0/2402175820 shutdown_connections 2023-10-06T13:53:19.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7ff7d000b640 1 --2- 172.21.15.19:0/2402175820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7c00c67c0 0x7ff7c00c6c00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7ff7d000b640 1 --2- 172.21.15.19:0/2402175820 >> v2:172.21.15.83:3300/0 conn(0x7ff7c0037260 0x7ff7c0034b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7ff7d000b640 1 -- 172.21.15.19:0/2402175820 >> 172.21.15.19:0/2402175820 conn(0x7ff7c0039ea0 msgr2=0x7ff7c003a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7fa799aaf640 1 -- start start 2023-10-06T13:53:19.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7fa799aaf640 1 librados: setting wanted keys 2023-10-06T13:53:19.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7fa799aaf640 1 librados: calling monclient init 2023-10-06T13:53:19.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7fa799aaf640 10 monclient: init 2023-10-06T13:53:19.369 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7ff7d000b640 1 -- 172.21.15.19:0/2402175820 shutdown_connections 2023-10-06T13:53:19.369 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7ff7d000b640 1 -- 172.21.15.19:0/2402175820 wait complete. 2023-10-06T13:53:19.369 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.320+0000 7fa799aaf640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.369 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 10 monclient: _add_conns ranks=[2,0,1] 2023-10-06T13:53:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa7940b00c0 0x7fa794144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 10 monclient(hunting): picked mon.noname-c con 0x7fa7940b00c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7940b3990 0x7fa79413d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 10 monclient(hunting): picked mon.noname-a con 0x7fa7940b3990 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7940b3d60 0x7fa79413da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 10 monclient(hunting): picked mon.noname-b con 0x7fa7940b3d60 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa794136ba0 con 0x7fa7940b3990 2023-10-06T13:53:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7940a90a0 con 0x7fa7940b3d60 2023-10-06T13:53:19.371 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7ff7d000b640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.371 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.371 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79413e190 con 0x7fa7940b00c0 2023-10-06T13:53:19.371 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.371 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7fa799aaf640 10 monclient(hunting): authenticate will time out at 1327.507542s 2023-10-06T13:53:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7ff7d000b640 10 monclient: monmap: 2023-10-06T13:53:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.373 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7ff7d000b640 1 librados: starting msgr at 2023-10-06T13:53:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7ff7d000b640 1 librados: starting objecter 2023-10-06T13:53:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7ff7d000b640 1 Processor -- start 2023-10-06T13:53:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7ff7d000b640 1 -- start start 2023-10-06T13:53:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7ff7d000b640 1 librados: setting wanted keys 2023-10-06T13:53:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7ff7d000b640 1 librados: calling monclient init 2023-10-06T13:53:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.321+0000 7ff7d000b640 10 monclient: init 2023-10-06T13:53:19.374 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7faae7e66640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.374 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7faae7e66640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.374 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7faae7e66640 10 monclient: monmap: 2023-10-06T13:53:19.374 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.374 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.374 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.323293+0000 2023-10-06T13:53:19.374 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.323293+0000 2023-10-06T13:53:19.375 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.375 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.375 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.375 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.375 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.375 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.375 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.375 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.376 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.376 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 10 monclient: _add_conns ranks=[2,0,1] 2023-10-06T13:53:19.376 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff7c00072c0 0x7ff7c00321f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.376 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 10 monclient(hunting): picked mon.noname-c con 0x7ff7c00072c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.376 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7c0037260 0x7ff7c015f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 10 monclient(hunting): picked mon.noname-a con 0x7ff7c0037260 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7c00c67c0 0x7ff7c015f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 10 monclient(hunting): picked mon.noname-b con 0x7ff7c00c67c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c00328b0 con 0x7ff7c0037260 2023-10-06T13:53:19.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c002f810 con 0x7ff7c00c67c0 2023-10-06T13:53:19.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c002fd30 con 0x7ff7c00072c0 2023-10-06T13:53:19.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.377 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ff7d000b640 10 monclient(hunting): authenticate will time out at 1327.508935s 2023-10-06T13:53:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa05affd640 1 -- 172.21.15.19:0/2691752570 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050031830 con 0x7fa05c13e2b0 2023-10-06T13:53:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa05affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa05affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa05affd640 10 monclient: dump: 2023-10-06T13:53:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.378 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.379 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa05affd640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa05affd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa05affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa05affd640 1 -- 172.21.15.19:0/2691752570 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa050031990 con 0x7fa05c13e2b0 2023-10-06T13:53:19.379 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa05affd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa05affd640 1 -- 172.21.15.19:0/2691752570 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050031d20 con 0x7fa05c13e2b0 2023-10-06T13:53:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa062884640 5 monclient: authenticate success, global_id 4309 2023-10-06T13:53:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa062884640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa062884640 4 set_mon_vals no callback set 2023-10-06T13:53:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa05affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa05affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7fa05affd640 10 monclient: dump: 2023-10-06T13:53:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.380 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.381 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ffb77fff640 1 --2- 172.21.15.19:0/1606106680 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffb7806da00 0x7ffb78137850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00c970 tx=0x7ffb6c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ffb74ff9640 10 client.?.objecter ms_handle_connect 0x7ffb7806da00 2023-10-06T13:53:19.381 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7ffb74ff9640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.382 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.322+0000 7faae7e66640 1 Processor -- start 2023-10-06T13:53:19.382 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa793fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7940b3d60 0x7fa79413da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.382 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7faae7e66640 1 -- start start 2023-10-06T13:53:19.382 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7faae7e66640 10 monclient: init 2023-10-06T13:53:19.383 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7ff7ce5a8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7c00c67c0 0x7ff7c015f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa062884640 10 monclient: shutdown 2023-10-06T13:53:19.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa062884640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa062884640 1 -- 172.21.15.19:0/2691752570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa05c13e2b0 msgr2=0x7fa05c13e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa062884640 1 --2- 172.21.15.19:0/2691752570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa05c13e2b0 0x7fa05c13e700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa0500097a0 tx=0x7fa05002b8d0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa062884640 1 -- 172.21.15.19:0/2691752570 shutdown_connections 2023-10-06T13:53:19.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa062884640 1 --2- 172.21.15.19:0/2691752570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa05c13e2b0 0x7fa05c13e700 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa062884640 1 -- 172.21.15.19:0/2691752570 >> 172.21.15.19:0/2691752570 conn(0x7fa05c0b4fa0 msgr2=0x7fa05c0b53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa062884640 1 -- 172.21.15.19:0/2691752570 shutdown_connections 2023-10-06T13:53:19.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa062884640 1 -- 172.21.15.19:0/2691752570 wait complete. 2023-10-06T13:53:19.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa793fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7940b3d60 0x7fa79413da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39354/0 (socket says 172.21.15.19:39354) 2023-10-06T13:53:19.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa793fff640 1 -- 172.21.15.19:0/3323333400 learned_addr learned my addr 172.21.15.19:0/3323333400 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa793fff640 10 monclient(hunting): get_auth_request con 0x7fa7940b3d60 auth_method 0 2023-10-06T13:53:19.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa793fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa793fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.323+0000 7fa793fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7faae7e66640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7faae7e66640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2023-10-06T13:53:19.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7faae7e66640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faae00b3410 0x7faae00af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7ff7ce5a8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7c00c67c0 0x7ff7c015f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39370/0 (socket says 172.21.15.19:39370) 2023-10-06T13:53:19.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7ff7ce5a8640 1 -- 172.21.15.19:0/1898729719 learned_addr learned my addr 172.21.15.19:0/1898729719 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7ff7ce5a8640 10 monclient(hunting): get_auth_request con 0x7ff7c00c67c0 auth_method 0 2023-10-06T13:53:19.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7ff7ce5a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7ff7ce5a8640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7ff7ce5a8640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.387 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7faae7e66640 10 monclient(hunting): picked mon.noname-b con 0x7faae00b3410 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.387 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7faae7e66640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faae00b37e0 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:19.387 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7faae7e66640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faae0145430 0x7faae0145890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.387 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7faae7e66640 10 monclient(hunting): picked mon.noname-a con 0x7faae0145430 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:19.388 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe094ed640 1 -- 172.21.15.19:0/2241411585 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc02c960 con 0x7fbe0413da90 2023-10-06T13:53:19.388 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe094ed640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.388 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe094ed640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:19.388 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe094ed640 10 monclient: dump: 2023-10-06T13:53:19.388 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.388 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.388 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.388 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.388 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.389 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.389 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.389 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.389 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.389 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.389 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe094ed640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.389 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe094ed640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.389 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe094ed640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.389 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe094ed640 1 -- 172.21.15.19:0/2241411585 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc02cac0 con 0x7fbe0413da90 2023-10-06T13:53:19.389 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe094ed640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.389 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe094ed640 1 -- 172.21.15.19:0/2241411585 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc02ce50 con 0x7fbe0413da90 2023-10-06T13:53:19.389 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe094ed640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe094ed640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:19.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe094ed640 10 monclient: dump: 2023-10-06T13:53:19.390 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.390 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.390 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.390 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.390 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.390 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.390 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.390 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.390 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.390 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe0c753640 5 monclient: authenticate success, global_id 4260 2023-10-06T13:53:19.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe0c753640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fbe0c753640 4 set_mon_vals no callback set 2023-10-06T13:53:19.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7faae7e66640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae00b3ce0 con 0x7faae00b37e0 2023-10-06T13:53:19.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7faae7e66640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7faae7e66640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae00ae900 con 0x7faae0145430 2023-10-06T13:53:19.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7faae7e66640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7faae7e66640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae00aeae0 con 0x7faae00b3410 2023-10-06T13:53:19.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fa062884640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7fa062884640 10 monclient: monmap: 2023-10-06T13:53:19.392 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.392 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.392 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.393 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.393 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.393 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.393 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.393 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.393 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.393 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7faae7e66640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.324+0000 7faae7e66640 10 monclient(hunting): authenticate will time out at 1327.510999s 2023-10-06T13:53:19.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fa062884640 1 librados: starting msgr at 2023-10-06T13:53:19.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fa062884640 1 librados: starting objecter 2023-10-06T13:53:19.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fbe0c753640 10 monclient: shutdown 2023-10-06T13:53:19.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fbe0c753640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fbe0c753640 1 -- 172.21.15.19:0/2241411585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe0413da90 msgr2=0x7fbe0413dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fbe0c753640 1 --2- 172.21.15.19:0/2241411585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe0413da90 0x7fbe0413dea0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbe040b0310 tx=0x7fbdfc02bdc0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fa062884640 1 Processor -- start 2023-10-06T13:53:19.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fbe0c753640 1 -- 172.21.15.19:0/2241411585 shutdown_connections 2023-10-06T13:53:19.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fbe0c753640 1 --2- 172.21.15.19:0/2241411585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe0413da90 0x7fbe0413dea0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fbe0c753640 1 --2- 172.21.15.19:0/2241411585 >> v2:172.21.15.83:3300/0 conn(0x7fbe040afd00 0x7fbe040a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fbe0c753640 1 -- 172.21.15.19:0/2241411585 >> 172.21.15.19:0/2241411585 conn(0x7fbe04065920 msgr2=0x7fbe04065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fa793fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fa793fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fa793fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fbe0c753640 1 -- 172.21.15.19:0/2241411585 shutdown_connections 2023-10-06T13:53:19.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fbe0c753640 1 -- 172.21.15.19:0/2241411585 wait complete. 2023-10-06T13:53:19.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fa062884640 1 -- start start 2023-10-06T13:53:19.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fa062884640 1 librados: setting wanted keys 2023-10-06T13:53:19.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fa062884640 1 librados: calling monclient init 2023-10-06T13:53:19.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.325+0000 7fa062884640 10 monclient: init 2023-10-06T13:53:19.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T13:53:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fbe0c753640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fbe0c753640 10 monclient: monmap: 2023-10-06T13:53:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.399 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.399 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa05c069e70 0x7fa05c0ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.399 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 10 monclient(hunting): picked mon.noname-b con 0x7fa05c069e70 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.399 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c13e2b0 0x7fa05c1d9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.400 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 10 monclient(hunting): picked mon.noname-a con 0x7fa05c13e2b0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.400 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa05c149790 0x7fa05c140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.400 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7ff7ce5a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.400 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7ff7ce5a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.400 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7ff7ce5a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 10 monclient(hunting): picked mon.noname-c con 0x7fa05c149790 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c1d90c0 con 0x7fa05c13e2b0 2023-10-06T13:53:19.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c1d93a0 con 0x7fa05c069e70 2023-10-06T13:53:19.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c1d9d70 con 0x7fa05c149790 2023-10-06T13:53:19.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fa062884640 10 monclient(hunting): authenticate will time out at 1327.513149s 2023-10-06T13:53:19.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fbe0c753640 1 librados: starting msgr at 2023-10-06T13:53:19.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.326+0000 7fbe0c753640 1 librados: starting objecter 2023-10-06T13:53:19.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 1 Processor -- start 2023-10-06T13:53:19.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 1 -- start start 2023-10-06T13:53:19.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 1 librados: setting wanted keys 2023-10-06T13:53:19.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 1 librados: calling monclient init 2023-10-06T13:53:19.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7f9e577fe640 1 --2- 172.21.15.19:0/1227251426 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9e50031580 0x7f9e500cebc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9e480098c0 tx=0x7f9e480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 10 monclient: init 2023-10-06T13:53:19.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7f9e54ff9640 10 client.?.objecter ms_handle_connect 0x7f9e50031580 2023-10-06T13:53:19.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7f9e54ff9640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T13:53:19.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe040afd00 0x7fbe04070240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 10 monclient(hunting): picked mon.noname-b con 0x7fbe040afd00 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe0413da90 0x7fbe0413e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 10 monclient(hunting): picked mon.noname-a con 0x7fbe0413da90 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbe041440a0 0x7fbe04131a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 10 monclient(hunting): picked mon.noname-c con 0x7fbe041440a0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe04070780 con 0x7fbe0413da90 2023-10-06T13:53:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0413fb90 con 0x7fbe040afd00 2023-10-06T13:53:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe04143230 con 0x7fbe041440a0 2023-10-06T13:53:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7fbe0c753640 10 monclient(hunting): authenticate will time out at 1327.514001s 2023-10-06T13:53:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.327+0000 7ffa9e7fc640 1 -- 172.21.15.19:0/3711152477 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9802cbf0 con 0x7ffaa00b00c0 2023-10-06T13:53:19.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffa9e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffa9e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:19.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffa9e7fc640 10 monclient: dump: 2023-10-06T13:53:19.407 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.407 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.407 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.407 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.407 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.407 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.407 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.408 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffa9e7fc640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffa9e7fc640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffa9e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffa9e7fc640 1 -- 172.21.15.19:0/3711152477 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9802cd50 con 0x7ffaa00b00c0 2023-10-06T13:53:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffaa567a640 5 monclient: authenticate success, global_id 4266 2023-10-06T13:53:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffaa567a640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffa9e7fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffa9e7fc640 1 -- 172.21.15.19:0/3711152477 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98034660 con 0x7ffaa00b00c0 2023-10-06T13:53:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffa9e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffa9e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:19.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffa9e7fc640 10 monclient: dump: 2023-10-06T13:53:19.409 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.409 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.409 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.409 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.409 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.409 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.409 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.409 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.409 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.409 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffaa567a640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffaa567a640 4 set_mon_vals no callback set 2023-10-06T13:53:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffaa567a640 10 monclient: shutdown 2023-10-06T13:53:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffaa567a640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffaa567a640 1 -- 172.21.15.19:0/3711152477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00b00c0 msgr2=0x7ffaa00b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffaa567a640 1 --2- 172.21.15.19:0/3711152477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00b00c0 0x7ffaa00b0520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0072330 tx=0x7ffa9802be40 comp rx=0 tx=0).stop 2023-10-06T13:53:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.328+0000 7ffaa567a640 1 -- 172.21.15.19:0/3711152477 shutdown_connections 2023-10-06T13:53:19.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffaa567a640 1 --2- 172.21.15.19:0/3711152477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00b00c0 0x7ffaa00b0520 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffaa567a640 1 --2- 172.21.15.19:0/3711152477 >> v2:172.21.15.83:3300/0 conn(0x7ffaa00b3990 0x7ffaa00af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffaa567a640 1 -- 172.21.15.19:0/3711152477 >> 172.21.15.19:0/3711152477 conn(0x7ffaa00b6e00 msgr2=0x7ffaa00b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffaa567a640 1 -- 172.21.15.19:0/3711152477 shutdown_connections 2023-10-06T13:53:19.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffaa567a640 1 -- 172.21.15.19:0/3711152477 wait complete. 2023-10-06T13:53:19.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7fa793fff640 10 monclient(hunting): handle_auth_done global_id 4315 payload 824 2023-10-06T13:53:19.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7fa793fff640 1 -- 172.21.15.19:0/3323333400 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa7940b00c0 msgr2=0x7fa794144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7fa793fff640 1 --2- 172.21.15.19:0/3323333400 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa7940b00c0 0x7fa794144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7fa793fff640 1 -- 172.21.15.19:0/3323333400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7940b3990 msgr2=0x7fa79413d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7fa793fff640 1 --2- 172.21.15.19:0/3323333400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7940b3990 0x7fa79413d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7fa793fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7fa793fff640 1 monclient: found mon.noname-b 2023-10-06T13:53:19.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7fa793fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7fa793fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7fa793fff640 1 -- 172.21.15.19:0/3323333400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7840094d0 con 0x7fa7940b3d60 2023-10-06T13:53:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb74ff9640 1 -- 172.21.15.19:0/1606106680 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c046070 con 0x7ffb7806da00 2023-10-06T13:53:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb74ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb74ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb74ff9640 10 monclient: dump: 2023-10-06T13:53:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.416 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb74ff9640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb74ff9640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb74ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb74ff9640 1 -- 172.21.15.19:0/1606106680 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c005b70 con 0x7ffb7806da00 2023-10-06T13:53:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb74ff9640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb74ff9640 1 -- 172.21.15.19:0/1606106680 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c00c120 con 0x7ffb7806da00 2023-10-06T13:53:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb74ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb74ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb74ff9640 10 monclient: dump: 2023-10-06T13:53:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.418 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb7d280640 5 monclient: authenticate success, global_id 4241 2023-10-06T13:53:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb7d280640 10 monclient: _renew_subs 2023-10-06T13:53:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb7d280640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb7d280640 1 -- 172.21.15.19:0/1606106680 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb781d17c0 con 0x7ffb7806da00 2023-10-06T13:53:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffb767fc640 4 set_mon_vals no callback set 2023-10-06T13:53:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffaa567a640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.330+0000 7ffb7d280640 10 client.4241.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.330+0000 7ffb7d280640 10 monclient: _renew_subs 2023-10-06T13:53:19.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.330+0000 7ffb7d280640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.330+0000 7ffb7d280640 1 -- 172.21.15.19:0/1606106680 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb780ad210 con 0x7ffb7806da00 2023-10-06T13:53:19.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.330+0000 7ffb7d280640 1 librados: init done 2023-10-06T13:53:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.329+0000 7ffaa567a640 10 monclient: monmap: 2023-10-06T13:53:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.421 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.421 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.421 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.330+0000 7ffaa567a640 1 librados: starting msgr at 2023-10-06T13:53:19.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.330+0000 7ffaa567a640 1 librados: starting objecter 2023-10-06T13:53:19.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.330+0000 7ffaa567a640 1 Processor -- start 2023-10-06T13:53:19.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.330+0000 7ffaa567a640 1 -- start start 2023-10-06T13:53:19.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.330+0000 7ffaa567a640 1 librados: setting wanted keys 2023-10-06T13:53:19.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.330+0000 7ffaa567a640 1 librados: calling monclient init 2023-10-06T13:53:19.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.330+0000 7ffaa567a640 10 monclient: init 2023-10-06T13:53:19.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffb7d280640 10 client.4241.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffb7d280640 15 client.4241.objecter _session_command_op_assign -1 1 2023-10-06T13:53:19.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffb7d280640 20 client.4241.objecter _get_session osd=-1 returning homeless 2023-10-06T13:53:19.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffb7d280640 10 client.4241.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffb7d280640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffb7d280640 1 -- 172.21.15.19:0/1606106680 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffb780b2ec0 con 0x7ffb7806da00 2023-10-06T13:53:19.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T13:53:19.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00b00c0 0x7ffaa00aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 10 monclient(hunting): picked mon.noname-a con 0x7ffaa00b00c0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa00b3990 0x7ffaa00ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 10 monclient(hunting): picked mon.noname-b con 0x7ffaa00b3990 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffaa00b3d60 0x7ffaa00acb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 10 monclient(hunting): picked mon.noname-c con 0x7ffaa00b3d60 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa00aab70 con 0x7ffaa00b00c0 2023-10-06T13:53:19.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa00ad0d0 con 0x7ffaa00b3990 2023-10-06T13:53:19.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa00a9bd0 con 0x7ffaa00b3d60 2023-10-06T13:53:19.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7ffaa567a640 10 monclient(hunting): authenticate will time out at 1327.518221s 2023-10-06T13:53:19.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.331+0000 7faae6403640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faae0145430 0x7faae0145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.332+0000 7ff7cd5a6640 1 --2- 172.21.15.19:0/1898729719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7c0037260 0x7ff7c015f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7ff7cd5a6640 10 monclient(hunting): get_auth_request con 0x7ff7c0037260 auth_method 0 2023-10-06T13:53:19.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7ff7cd5a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7ff7cd5a6640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7ff7cd5a6640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7faae6403640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faae0145430 0x7faae0145890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38804/0 (socket says 172.21.15.19:38804) 2023-10-06T13:53:19.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7faae6403640 1 -- 172.21.15.19:0/1416656544 learned_addr learned my addr 172.21.15.19:0/1416656544 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7faae6403640 10 monclient(hunting): get_auth_request con 0x7faae0145430 auth_method 0 2023-10-06T13:53:19.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7faae6403640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7faae6403640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7faae6403640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7ff7ce5a8640 10 monclient(hunting): handle_auth_done global_id 4318 payload 824 2023-10-06T13:53:19.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7ff7ce5a8640 1 -- 172.21.15.19:0/1898729719 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff7c00072c0 msgr2=0x7ff7c00321f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7ff7ce5a8640 1 --2- 172.21.15.19:0/1898729719 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff7c00072c0 0x7ff7c00321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7ff7ce5a8640 1 -- 172.21.15.19:0/1898729719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7c0037260 msgr2=0x7ff7c015f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7ff7ce5a8640 1 --2- 172.21.15.19:0/1898729719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7c0037260 0x7ff7c015f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7ff7ce5a8640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7ff7ce5a8640 1 monclient: found mon.noname-b 2023-10-06T13:53:19.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7ff7ce5a8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7ff7ce5a8640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.333+0000 7ff7ce5a8640 1 -- 172.21.15.19:0/1898729719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c8066930 con 0x7ff7c00c67c0 2023-10-06T13:53:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.335+0000 7faae6403640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.335+0000 7faae6403640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.335+0000 7faae6403640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.337+0000 7faae6403640 10 monclient(hunting): handle_auth_done global_id 4275 payload 274 2023-10-06T13:53:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.337+0000 7faae6403640 1 -- 172.21.15.19:0/1416656544 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faae00b3410 msgr2=0x7faae00af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.337+0000 7faae6403640 1 --2- 172.21.15.19:0/1416656544 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faae00b3410 0x7faae00af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.337+0000 7faae6403640 1 -- 172.21.15.19:0/1416656544 >> v1:172.21.15.83:6789/0 conn(0x7faae00b37e0 legacy=0x7faae00afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.337+0000 7faae6403640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.337+0000 7faae6403640 1 monclient: found mon.noname-a 2023-10-06T13:53:19.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.337+0000 7faae6403640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.337+0000 7faae6403640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.337+0000 7faae6403640 1 -- 172.21.15.19:0/1416656544 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae00aedc0 con 0x7faae0145430 2023-10-06T13:53:19.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.338+0000 7ff7ce5a8640 1 --2- 172.21.15.19:0/1898729719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7c00c67c0 0x7ff7c015f970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff7c804f270 tx=0x7ff7c8068560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.338+0000 7ff7be7fc640 10 client.?.objecter ms_handle_connect 0x7ff7c00c67c0 2023-10-06T13:53:19.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.338+0000 7ff7be7fc640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.338+0000 7fa793fff640 1 --2- 172.21.15.19:0/3323333400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7940b3d60 0x7fa79413da30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00b9a0 tx=0x7fa78c00be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.338+0000 7fa790ff9640 10 client.?.objecter ms_handle_connect 0x7fa7940b3d60 2023-10-06T13:53:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.338+0000 7fa790ff9640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.339+0000 7fbe09cee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe0413da90 0x7fbe0413e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.340+0000 7fa05bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa05c069e70 0x7fa05c0ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.341+0000 7faae6403640 1 --2- 172.21.15.19:0/1416656544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae0145430 0x7faae0145890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faad8009e50 tx=0x7faad802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbba9b11617b098 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e54ff9640 1 -- 172.21.15.19:0/1227251426 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48033c70 con 0x7f9e50031580 2023-10-06T13:53:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e54ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7fbe09cee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe0413da90 0x7fbe0413e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38824/0 (socket says 172.21.15.19:38824) 2023-10-06T13:53:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7fbe09cee640 1 -- 172.21.15.19:0/643282779 learned_addr learned my addr 172.21.15.19:0/643282779 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7fbe09cee640 10 monclient(hunting): get_auth_request con 0x7fbe0413da90 auth_method 0 2023-10-06T13:53:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7fbe09cee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e54ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e54ff9640 10 monclient: dump: 2023-10-06T13:53:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.434 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.434 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.434 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.434 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.434 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.434 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.434 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e54ff9640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e54ff9640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e54ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7fbe09cee640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7fbe09cee640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e5e3f5640 5 monclient: authenticate success, global_id 4244 2023-10-06T13:53:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e5e3f5640 10 monclient: _renew_subs 2023-10-06T13:53:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e5e3f5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e5e3f5640 1 -- 172.21.15.19:0/1227251426 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5015dee0 con 0x7f9e50031580 2023-10-06T13:53:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e54ff9640 1 -- 172.21.15.19:0/1227251426 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e48033dd0 con 0x7f9e50031580 2023-10-06T13:53:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e54ff9640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e54ff9640 1 -- 172.21.15.19:0/1227251426 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4803e6c0 con 0x7f9e50031580 2023-10-06T13:53:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e54ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e54ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e54ff9640 10 monclient: dump: 2023-10-06T13:53:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.437 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e567fc640 4 set_mon_vals no callback set 2023-10-06T13:53:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e5e3f5640 10 client.4244.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e5e3f5640 10 monclient: _renew_subs 2023-10-06T13:53:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e5e3f5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.438 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e5e3f5640 1 -- 172.21.15.19:0/1227251426 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e500cf930 con 0x7f9e50031580 2023-10-06T13:53:19.438 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7f9e5e3f5640 1 librados: init done 2023-10-06T13:53:19.438 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7fa060e21640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa05c149790 0x7fa05c140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.438 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.342+0000 7ffa9effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00b00c0 0x7ffaa00aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7f9e5e3f5640 10 client.4244.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7fa05b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c13e2b0 0x7fa05c1d9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7f9e5e3f5640 15 client.4244.objecter _session_command_op_assign -1 1 2023-10-06T13:53:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7f9e5e3f5640 20 client.4244.objecter _get_session osd=-1 returning homeless 2023-10-06T13:53:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7f9e5e3f5640 10 client.4244.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7f9e5e3f5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7f9e5e3f5640 1 -- 172.21.15.19:0/1227251426 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9e50032b00 con 0x7f9e50031580 2023-10-06T13:53:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7ffa9effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00b00c0 0x7ffaa00aa0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38834/0 (socket says 172.21.15.19:38834) 2023-10-06T13:53:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7ffa9effd640 1 -- 172.21.15.19:0/2166371806 learned_addr learned my addr 172.21.15.19:0/2166371806 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7ffa9effd640 10 monclient(hunting): get_auth_request con 0x7ffaa00b00c0 auth_method 0 2023-10-06T13:53:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7ffa9effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.441 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7ffa9effd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.441 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7ffa9effd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.441 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7fa060e21640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa05c149790 0x7fa05c140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:41288/0 (socket says 172.21.15.19:41288) 2023-10-06T13:53:19.441 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7fa060e21640 1 -- 172.21.15.19:0/3510967138 learned_addr learned my addr 172.21.15.19:0/3510967138 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.441 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7fa060e21640 10 monclient(hunting): get_auth_request con 0x7fa05c149790 auth_method 0 2023-10-06T13:53:19.441 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7fa060e21640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.441 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7fa060e21640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.441 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.343+0000 7fa060e21640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.442 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.344+0000 7fbe0a4ef640 1 --2- 172.21.15.19:0/643282779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe040afd00 0x7fbe04070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.442 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.344+0000 7fbe09cee640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.442 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.344+0000 7fbe09cee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.442 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.344+0000 7fbe09cee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.345+0000 7fa060e21640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.345+0000 7fa060e21640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.345+0000 7fa060e21640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.345+0000 7fbe0a4ef640 10 monclient(hunting): get_auth_request con 0x7fbe040afd00 auth_method 0 2023-10-06T13:53:19.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.345+0000 7fbe0a4ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.345+0000 7fbe0a4ef640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.345+0000 7fbe0a4ef640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.444 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.345+0000 7fa05bfff640 10 monclient(hunting): get_auth_request con 0x7fa05c069e70 auth_method 0 2023-10-06T13:53:19.444 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.345+0000 7fa05bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.444 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.345+0000 7fa05bfff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.444 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.345+0000 7fa05bfff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.444 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.346+0000 7fa05bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.346+0000 7fa05bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.346+0000 7fa05bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.346+0000 7fbe0acf0640 1 --2- 172.21.15.19:0/643282779 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbe041440a0 0x7fbe04131a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.346+0000 7fbe0a4ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.346+0000 7fbe0a4ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.346+0000 7fbe0a4ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7fbe09cee640 10 monclient(hunting): handle_auth_done global_id 4278 payload 824 2023-10-06T13:53:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7fbe09cee640 1 -- 172.21.15.19:0/643282779 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbe041440a0 msgr2=0x7fbe04131a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7fbe09cee640 1 --2- 172.21.15.19:0/643282779 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbe041440a0 0x7fbe04131a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7fbe09cee640 1 -- 172.21.15.19:0/643282779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe040afd00 msgr2=0x7fbe04070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7fbe09cee640 1 --2- 172.21.15.19:0/643282779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe040afd00 0x7fbe04070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7fbe09cee640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7fbe09cee640 1 monclient: found mon.noname-a 2023-10-06T13:53:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7fbe09cee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7fbe09cee640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7fbe09cee640 1 -- 172.21.15.19:0/643282779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdfc009b30 con 0x7fbe0413da90 2023-10-06T13:53:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7fa05b7fe640 10 monclient(hunting): get_auth_request con 0x7fa05c13e2b0 auth_method 0 2023-10-06T13:53:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7fa05b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7fa05b7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7fa05b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.347+0000 7ffa967fc640 1 --2- 172.21.15.19:0/2166371806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa00b3990 0x7ffaa00ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7be7fc640 1 -- 172.21.15.19:0/1898729719 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8060ab0 con 0x7ff7c00c67c0 2023-10-06T13:53:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7be7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7be7fc640 10 monclient: dump: 2023-10-06T13:53:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.450 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7be7fc640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7be7fc640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7be7fc640 1 -- 172.21.15.19:0/1898729719 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c8060c10 con 0x7ff7c00c67c0 2023-10-06T13:53:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7be7fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7be7fc640 1 -- 172.21.15.19:0/1898729719 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c80b7410 con 0x7ff7c00c67c0 2023-10-06T13:53:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7d000b640 5 monclient: authenticate success, global_id 4318 2023-10-06T13:53:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7d000b640 10 monclient: _renew_subs 2023-10-06T13:53:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7d000b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7d000b640 1 -- 172.21.15.19:0/1898729719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c002f330 con 0x7ff7c00c67c0 2023-10-06T13:53:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7bffff640 4 set_mon_vals no callback set 2023-10-06T13:53:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7be7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7be7fc640 10 monclient: dump: 2023-10-06T13:53:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.452 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7d000b640 10 client.4318.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7d000b640 10 monclient: _renew_subs 2023-10-06T13:53:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7d000b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7d000b640 1 -- 172.21.15.19:0/1898729719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c002f5e0 con 0x7ff7c00c67c0 2023-10-06T13:53:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.348+0000 7ff7d000b640 1 librados: init done 2023-10-06T13:53:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.349+0000 7ff7d000b640 10 client.4318.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.349+0000 7ff7d000b640 15 client.4318.objecter _session_command_op_assign -1 1 2023-10-06T13:53:19.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.349+0000 7ff7d000b640 20 client.4318.objecter _get_session osd=-1 returning homeless 2023-10-06T13:53:19.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.349+0000 7ff7d000b640 10 client.4318.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.349+0000 7ff7d000b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.349+0000 7ff7d000b640 1 -- 172.21.15.19:0/1898729719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff7c0036b60 con 0x7ff7c00c67c0 2023-10-06T13:53:19.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.350+0000 7ffa9effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.350+0000 7ffa9effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.350+0000 7ffa9effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.351+0000 7fa05b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.352+0000 7fa05b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.352+0000 7fa05b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.355+0000 7fa05b7fe640 10 monclient(hunting): handle_auth_done global_id 4284 payload 824 2023-10-06T13:53:19.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.355+0000 7fa05b7fe640 1 -- 172.21.15.19:0/3510967138 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa05c149790 msgr2=0x7fa05c140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.355+0000 7fa05b7fe640 1 --2- 172.21.15.19:0/3510967138 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa05c149790 0x7fa05c140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.355+0000 7fa05b7fe640 1 -- 172.21.15.19:0/3510967138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa05c069e70 msgr2=0x7fa05c0ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.355+0000 7fa05b7fe640 1 --2- 172.21.15.19:0/3510967138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa05c069e70 0x7fa05c0ad1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.355+0000 7fa05b7fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.355+0000 7fa05b7fe640 1 monclient: found mon.noname-a 2023-10-06T13:53:19.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.355+0000 7fa05b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.355+0000 7fa05b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.355+0000 7fa05b7fe640 1 -- 172.21.15.19:0/3510967138 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa050009400 con 0x7fa05c13e2b0 2023-10-06T13:53:19.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.359+0000 7ffa9f7fe640 1 --2- 172.21.15.19:0/2166371806 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffaa00b3d60 0x7ffaa00acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.456 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.360+0000 7fbe09cee640 1 --2- 172.21.15.19:0/643282779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe0413da90 0x7fbe0413e920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc036040 tx=0x7fbdfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.456 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.360+0000 7fbdfaffd640 10 client.?.objecter ms_handle_connect 0x7fbe0413da90 2023-10-06T13:53:19.456 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.360+0000 7fbdfaffd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.360+0000 7ffa9effd640 10 monclient(hunting): handle_auth_done global_id 4281 payload 824 2023-10-06T13:53:19.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.360+0000 7ffa9effd640 1 -- 172.21.15.19:0/2166371806 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffaa00b3d60 msgr2=0x7ffaa00acb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.360+0000 7ffa9effd640 1 --2- 172.21.15.19:0/2166371806 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffaa00b3d60 0x7ffaa00acb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.360+0000 7ffa9effd640 1 -- 172.21.15.19:0/2166371806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa00b3990 msgr2=0x7ffaa00ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.360+0000 7ffa9effd640 1 --2- 172.21.15.19:0/2166371806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa00b3990 0x7ffaa00ac650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.360+0000 7ffa9effd640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.360+0000 7ffa9effd640 1 monclient: found mon.noname-a 2023-10-06T13:53:19.457 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.360+0000 7ffa9effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.458 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.360+0000 7ffa9effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.458 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.360+0000 7ffa9effd640 1 -- 172.21.15.19:0/2166371806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa98009a70 con 0x7ffaa00b00c0 2023-10-06T13:53:19.459 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa790ff9640 1 -- 172.21.15.19:0/3323333400 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00cba0 con 0x7fa7940b3d60 2023-10-06T13:53:19.459 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa790ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.459 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa790ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:19.459 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa790ff9640 10 monclient: dump: 2023-10-06T13:53:19.459 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.459 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.459 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.459 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.459 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.459 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.460 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa790ff9640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa790ff9640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa790ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa790ff9640 1 -- 172.21.15.19:0/3323333400 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c00cd20 con 0x7fa7940b3d60 2023-10-06T13:53:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa790ff9640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa790ff9640 1 -- 172.21.15.19:0/3323333400 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c014680 con 0x7fa7940b3d60 2023-10-06T13:53:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa790ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa790ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa790ff9640 10 monclient: dump: 2023-10-06T13:53:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.461 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa799aaf640 5 monclient: authenticate success, global_id 4315 2023-10-06T13:53:19.462 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa799aaf640 10 monclient: _renew_subs 2023-10-06T13:53:19.462 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa799aaf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.462 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa7927fc640 4 set_mon_vals no callback set 2023-10-06T13:53:19.462 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa799aaf640 1 -- 172.21.15.19:0/3323333400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa794144c80 con 0x7fa7940b3d60 2023-10-06T13:53:19.462 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa799aaf640 10 client.4315.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.462 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa799aaf640 10 monclient: _renew_subs 2023-10-06T13:53:19.462 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa799aaf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.462 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa799aaf640 1 -- 172.21.15.19:0/3323333400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7940a9630 con 0x7fa7940b3d60 2023-10-06T13:53:19.462 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.362+0000 7fa799aaf640 1 librados: init done 2023-10-06T13:53:19.462 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.363+0000 7fa05b7fe640 1 --2- 172.21.15.19:0/3510967138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c13e2b0 0x7fa05c1d9830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa05c06e050 tx=0x7fa0500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.463 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.363+0000 7fa058ff9640 10 client.?.objecter ms_handle_connect 0x7fa05c13e2b0 2023-10-06T13:53:19.463 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.363+0000 7fa058ff9640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.463 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.363+0000 7fa799aaf640 10 client.4315.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.463 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.363+0000 7fa799aaf640 15 client.4315.objecter _session_command_op_assign -1 1 2023-10-06T13:53:19.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.363+0000 7fa799aaf640 20 client.4315.objecter _get_session osd=-1 returning homeless 2023-10-06T13:53:19.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.363+0000 7fa799aaf640 10 client.4315.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.363+0000 7fa799aaf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.363+0000 7fa799aaf640 1 -- 172.21.15.19:0/3323333400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa7940af110 con 0x7fa7940b3d60 2023-10-06T13:53:19.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.370+0000 7ffa9effd640 1 --2- 172.21.15.19:0/2166371806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00b00c0 0x7ffaa00aa0d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000c7f0 tx=0x7ffa9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.370+0000 7ffa97fff640 10 client.?.objecter ms_handle_connect 0x7ffaa00b00c0 2023-10-06T13:53:19.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.370+0000 7ffa97fff640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.465 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.371+0000 7fbdfaffd640 1 -- 172.21.15.19:0/643282779 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc03e450 con 0x7fbe0413da90 2023-10-06T13:53:19.465 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbdfaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.465 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbdfaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:19.465 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbdfaffd640 10 monclient: dump: 2023-10-06T13:53:19.465 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.466 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.466 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.466 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.466 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.466 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.466 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.466 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.466 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.466 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbdfaffd640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbdfaffd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbdfaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbdfaffd640 1 -- 172.21.15.19:0/643282779 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc03e5d0 con 0x7fbe0413da90 2023-10-06T13:53:19.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbdfaffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbdfaffd640 1 -- 172.21.15.19:0/643282779 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc03e980 con 0x7fbe0413da90 2023-10-06T13:53:19.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbdfaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbdfaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:19.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbdfaffd640 10 monclient: dump: 2023-10-06T13:53:19.467 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.467 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.467 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.467 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.467 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.468 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.468 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.468 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.468 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.468 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.468 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 5 monclient: authenticate success, global_id 4278 2023-10-06T13:53:19.468 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe08cec640 4 set_mon_vals no callback set 2023-10-06T13:53:19.468 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 10 monclient: _renew_subs 2023-10-06T13:53:19.468 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.468 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 1 -- 172.21.15.19:0/643282779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe04133360 con 0x7fbe0413da90 2023-10-06T13:53:19.468 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 10 client.4278.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.468 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 10 monclient: _renew_subs 2023-10-06T13:53:19.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 1 -- 172.21.15.19:0/643282779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0414ba40 con 0x7fbe0413da90 2023-10-06T13:53:19.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 1 librados: init done 2023-10-06T13:53:19.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 10 client.4278.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 15 client.4278.objecter _session_command_op_assign -1 1 2023-10-06T13:53:19.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 20 client.4278.objecter _get_session osd=-1 returning homeless 2023-10-06T13:53:19.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 10 client.4278.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.372+0000 7fbe0c753640 1 -- 172.21.15.19:0/643282779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe0413e3e0 con 0x7fbe0413da90 2023-10-06T13:53:19.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae4c00640 1 -- 172.21.15.19:0/1416656544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad802ca70 con 0x7faae0145430 2023-10-06T13:53:19.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae4c00640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae4c00640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:19.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae4c00640 10 monclient: dump: 2023-10-06T13:53:19.470 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.470 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.470 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.470 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.471 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.471 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.471 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.471 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.471 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.471 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae4c00640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae4c00640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae4c00640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae4c00640 1 -- 172.21.15.19:0/1416656544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad802cbd0 con 0x7faae0145430 2023-10-06T13:53:19.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae4c00640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.471 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae4c00640 1 -- 172.21.15.19:0/1416656544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad80343f0 con 0x7faae0145430 2023-10-06T13:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae4c00640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae4c00640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae4c00640 10 monclient: dump: 2023-10-06T13:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.473 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae7e66640 5 monclient: authenticate success, global_id 4275 2023-10-06T13:53:19.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae7e66640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.374+0000 7faae7e66640 4 set_mon_vals no callback set 2023-10-06T13:53:19.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.375+0000 7ff7be7fc640 1 -- 172.21.15.19:0/1898729719 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff7c80b4050 con 0x7ff7c00c67c0 2023-10-06T13:53:19.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.375+0000 7faae7e66640 10 monclient: shutdown 2023-10-06T13:53:19.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.375+0000 7faae7e66640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.375+0000 7faae7e66640 1 -- 172.21.15.19:0/1416656544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae0145430 msgr2=0x7faae0145890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.375+0000 7faae7e66640 1 --2- 172.21.15.19:0/1416656544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae0145430 0x7faae0145890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faad8009e50 tx=0x7faad802bed0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.375+0000 7faae7e66640 1 -- 172.21.15.19:0/1416656544 shutdown_connections 2023-10-06T13:53:19.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.375+0000 7faae7e66640 1 --2- 172.21.15.19:0/1416656544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae0145430 0x7faae0145890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.375+0000 7faae7e66640 1 --2- 172.21.15.19:0/1416656544 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faae00b3410 0x7faae00af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.375+0000 7faae7e66640 1 -- 172.21.15.19:0/1416656544 >> 172.21.15.19:0/1416656544 conn(0x7faae00b6e00 msgr2=0x7faae00b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.375+0000 7faae7e66640 1 -- 172.21.15.19:0/1416656544 shutdown_connections 2023-10-06T13:53:19.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.375+0000 7ff7be7fc640 1 --2- 172.21.15.19:0/1898729719 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ff7ac06c210 0x7ff7ac06e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.375+0000 7faae7e66640 1 -- 172.21.15.19:0/1416656544 wait complete. 2023-10-06T13:53:19.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.376+0000 7ff7cdda7640 1 --2- 172.21.15.19:0/1898729719 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ff7ac06c210 0x7ff7ac06e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.376+0000 7ff7cdda7640 10 monclient: get_auth_request con 0x7ff7ac06c210 auth_method 0 2023-10-06T13:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.376+0000 7faae7e66640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.376+0000 7faae7e66640 10 monclient: monmap: 2023-10-06T13:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.376+0000 7ff7be7fc640 10 client.4318.objecter ms_handle_connect 0x7ff7ac06c210 2023-10-06T13:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.376+0000 7ff7cdda7640 1 --2- 172.21.15.19:0/1898729719 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ff7ac06c210 0x7ff7ac06e6c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4001560 tx=0x7ff7c4007990 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.376+0000 7faae7e66640 1 librados: starting msgr at 2023-10-06T13:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.376+0000 7faae7e66640 1 librados: starting objecter 2023-10-06T13:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.376+0000 7faae7e66640 1 Processor -- start 2023-10-06T13:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 1 -- start start 2023-10-06T13:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 1 librados: setting wanted keys 2023-10-06T13:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 1 librados: calling monclient init 2023-10-06T13:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 10 monclient: init 2023-10-06T13:53:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:53:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae00b3410 0x7faae01434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 10 monclient(hunting): picked mon.noname-a con 0x7faae00b3410 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faae00b37e0 0x7faae01dd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 10 monclient(hunting): picked mon.noname-c con 0x7faae00b37e0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 1 -- 172.21.15.19:0/1898729719 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff7c80f4bd0 con 0x7ff7c00c67c0 2023-10-06T13:53:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 10 client.4318.objecter ms_dispatch 0x7ff7c8000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 3 client.4318.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 3 client.4318.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae0145430 0x7faae01cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa058ff9640 1 -- 172.21.15.19:0/3510967138 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05003ba70 con 0x7fa05c13e2b0 2023-10-06T13:53:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa058ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa058ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa058ff9640 10 monclient: dump: 2023-10-06T13:53:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.482 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa058ff9640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa058ff9640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa058ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa058ff9640 1 -- 172.21.15.19:0/3510967138 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05003bbf0 con 0x7fa05c13e2b0 2023-10-06T13:53:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa058ff9640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa058ff9640 1 -- 172.21.15.19:0/3510967138 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0500433f0 con 0x7fa05c13e2b0 2023-10-06T13:53:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa058ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa058ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa058ff9640 10 monclient: dump: 2023-10-06T13:53:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.483 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa062884640 5 monclient: authenticate success, global_id 4284 2023-10-06T13:53:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa062884640 10 monclient: _renew_subs 2023-10-06T13:53:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa062884640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa062884640 1 -- 172.21.15.19:0/3510967138 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05c13ff10 con 0x7fa05c13e2b0 2023-10-06T13:53:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 10 client.4318.objecter checking command 1 2023-10-06T13:53:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 1 --2- 172.21.15.19:0/1898729719 >> [v2:172.21.15.83:6826/86123,v1:172.21.15.83:6827/86123] conn(0x7ff7ac0725e0 0x7ff7ac0749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 20 client.4318.objecter _get_session s=0x7ff7ac071f10 osd=4 3 2023-10-06T13:53:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 20 client.4318.objecter put_session s=0x7ff7ac071f10 osd=4 3 2023-10-06T13:53:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 15 client.4318.objecter _session_command_op_remove -1 1 2023-10-06T13:53:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 20 client.4318.objecter _get_session s=0x7ff7ac071f10 osd=4 3 2023-10-06T13:53:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 20 client.4318.objecter get_session s=0x7ff7ac071f10 osd=4 3 2023-10-06T13:53:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 15 client.4318.objecter _session_command_op_assign 4 1 2023-10-06T13:53:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 20 client.4318.objecter put_session s=0x7ff7ac071f10 osd=4 4 2023-10-06T13:53:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 10 client.4318.objecter _send_command 1 2023-10-06T13:53:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 1 -- 172.21.15.19:0/1898729719 --> [v2:172.21.15.83:6826/86123,v1:172.21.15.83:6827/86123] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7ac0750a0 con 0x7ff7ac0725e0 2023-10-06T13:53:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7ff7be7fc640 20 client.4318.objecter dump_active .. 0 homeless 2023-10-06T13:53:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7fa059ffb640 4 set_mon_vals no callback set 2023-10-06T13:53:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7fa062884640 10 client.4284.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7fa062884640 10 monclient: _renew_subs 2023-10-06T13:53:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7fa062884640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7fa062884640 1 -- 172.21.15.19:0/3510967138 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05c1411f0 con 0x7fa05c13e2b0 2023-10-06T13:53:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 10 monclient(hunting): picked mon.noname-b con 0x7faae0145430 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae01ce550 con 0x7faae00b3410 2023-10-06T13:53:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae01cc520 con 0x7faae0145430 2023-10-06T13:53:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae01cc690 con 0x7faae00b37e0 2023-10-06T13:53:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.377+0000 7faae7e66640 10 monclient(hunting): authenticate will time out at 1327.564219s 2023-10-06T13:53:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7ff7be7fc640 1 -- 172.21.15.19:0/1898729719 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7c80f0810 con 0x7ff7c00c67c0 2023-10-06T13:53:19.487 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7ff7be7fc640 10 monclient: handle_get_version_reply finishing 1 version 23 2023-10-06T13:53:19.488 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7fa062884640 1 librados: init done 2023-10-06T13:53:19.488 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7ff7cd5a6640 1 --2- 172.21.15.19:0/1898729719 >> [v2:172.21.15.83:6826/86123,v1:172.21.15.83:6827/86123] conn(0x7ff7ac0725e0 0x7ff7ac0749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.488 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7ffb74ff9640 1 -- 172.21.15.19:0/1606106680 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ffb6c00c280 con 0x7ffb7806da00 2023-10-06T13:53:19.488 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7ffb74ff9640 1 --2- 172.21.15.19:0/1606106680 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ffb6406c210 0x7ffb6406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.488 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7fa062884640 10 client.4284.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7fa062884640 15 client.4284.objecter _session_command_op_assign -1 1 2023-10-06T13:53:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7fa062884640 20 client.4284.objecter _get_session osd=-1 returning homeless 2023-10-06T13:53:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7fa062884640 10 client.4284.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7fa062884640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.378+0000 7fa062884640 1 -- 172.21.15.19:0/3510967138 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa05c0ab9c0 con 0x7fa05c13e2b0 2023-10-06T13:53:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.379+0000 7ff7cd5a6640 10 monclient: get_auth_request con 0x7ff7ac0725e0 auth_method 0 2023-10-06T13:53:19.490 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.379+0000 7ffb777fe640 1 --2- 172.21.15.19:0/1606106680 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ffb6406c210 0x7ffb6406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.490 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.379+0000 7ffb777fe640 10 monclient: get_auth_request con 0x7ffb6406c210 auth_method 0 2023-10-06T13:53:19.490 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.379+0000 7ffb74ff9640 10 client.4241.objecter ms_handle_connect 0x7ffb6406c210 2023-10-06T13:53:19.490 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.379+0000 7ffb777fe640 1 --2- 172.21.15.19:0/1606106680 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ffb6406c210 0x7ffb6406e6c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffb700070b0 tx=0x7ffb70007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.490 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.382+0000 7ff7cd5a6640 1 --2- 172.21.15.19:0/1898729719 >> [v2:172.21.15.83:6826/86123,v1:172.21.15.83:6827/86123] conn(0x7ff7ac0725e0 0x7ff7ac0749f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.490 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.382+0000 7ff7be7fc640 10 client.4318.objecter ms_handle_connect 0x7ff7ac0725e0 2023-10-06T13:53:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffa97fff640 1 -- 172.21.15.19:0/2166371806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa90004220 con 0x7ffaa00b00c0 2023-10-06T13:53:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffa97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffa97fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffa97fff640 10 monclient: dump: 2023-10-06T13:53:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.493 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffa97fff640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffa97fff640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffa97fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffa97fff640 1 -- 172.21.15.19:0/2166371806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa900043a0 con 0x7ffaa00b00c0 2023-10-06T13:53:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffaa567a640 5 monclient: authenticate success, global_id 4281 2023-10-06T13:53:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffaa567a640 10 monclient: _renew_subs 2023-10-06T13:53:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffaa567a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffaa567a640 1 -- 172.21.15.19:0/2166371806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa0144d80 con 0x7ffaa00b00c0 2023-10-06T13:53:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffa97fff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffaa567a640 10 client.4281.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffa9dffb640 4 set_mon_vals no callback set 2023-10-06T13:53:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffaa567a640 10 monclient: _renew_subs 2023-10-06T13:53:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffaa567a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffaa567a640 1 -- 172.21.15.19:0/2166371806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa00ab000 con 0x7ffaa00b00c0 2023-10-06T13:53:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffaa567a640 1 librados: init done 2023-10-06T13:53:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffa97fff640 1 -- 172.21.15.19:0/2166371806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa90007860 con 0x7ffaa00b00c0 2023-10-06T13:53:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffa97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffa97fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.389+0000 7ffa97fff640 10 monclient: dump: 2023-10-06T13:53:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.495 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.390+0000 7ffaa567a640 10 client.4281.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.390+0000 7ffaa567a640 15 client.4281.objecter _session_command_op_assign -1 1 2023-10-06T13:53:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.390+0000 7ffaa567a640 20 client.4281.objecter _get_session osd=-1 returning homeless 2023-10-06T13:53:19.496 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.390+0000 7ffaa567a640 10 client.4281.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.496 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.390+0000 7ffaa567a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.496 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.390+0000 7ffaa567a640 1 -- 172.21.15.19:0/2166371806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffaa00af110 con 0x7ffaa00b00c0 2023-10-06T13:53:19.496 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.394+0000 7ffb74ff9640 1 -- 172.21.15.19:0/1606106680 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ffb6c07aa20 con 0x7ffb7806da00 2023-10-06T13:53:19.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.394+0000 7ffb74ff9640 10 client.4241.objecter ms_dispatch 0x7ffb78000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:19.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.394+0000 7ffb74ff9640 3 client.4241.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:19.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.394+0000 7ffb74ff9640 3 client.4241.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:19.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb74ff9640 10 client.4241.objecter checking command 1 2023-10-06T13:53:19.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb74ff9640 1 --2- 172.21.15.19:0/1606106680 >> [v2:172.21.15.19:6808/86194,v1:172.21.15.19:6809/86194] conn(0x7ffb640725e0 0x7ffb640749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb74ff9640 20 client.4241.objecter _get_session s=0x7ffb64071f10 osd=2 3 2023-10-06T13:53:19.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb74ff9640 20 client.4241.objecter put_session s=0x7ffb64071f10 osd=2 3 2023-10-06T13:53:19.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb74ff9640 15 client.4241.objecter _session_command_op_remove -1 1 2023-10-06T13:53:19.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb74ff9640 20 client.4241.objecter _get_session s=0x7ffb64071f10 osd=2 3 2023-10-06T13:53:19.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb74ff9640 20 client.4241.objecter get_session s=0x7ffb64071f10 osd=2 3 2023-10-06T13:53:19.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb74ff9640 15 client.4241.objecter _session_command_op_assign 2 1 2023-10-06T13:53:19.498 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb74ff9640 20 client.4241.objecter put_session s=0x7ffb64071f10 osd=2 4 2023-10-06T13:53:19.498 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb74ff9640 10 client.4241.objecter _send_command 1 2023-10-06T13:53:19.498 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb74ff9640 1 -- 172.21.15.19:0/1606106680 --> [v2:172.21.15.19:6808/86194,v1:172.21.15.19:6809/86194] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb640750a0 con 0x7ffb640725e0 2023-10-06T13:53:19.498 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb74ff9640 20 client.4241.objecter dump_active .. 0 homeless 2023-10-06T13:53:19.498 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb74ff9640 1 -- 172.21.15.19:0/1606106680 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb6c076d60 con 0x7ffb7806da00 2023-10-06T13:53:19.498 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb74ff9640 10 monclient: handle_get_version_reply finishing 1 version 23 2023-10-06T13:53:19.498 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7f9e54ff9640 1 -- 172.21.15.19:0/1227251426 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f9e4803a040 con 0x7f9e50031580 2023-10-06T13:53:19.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7ffb7ca7f640 1 --2- 172.21.15.19:0/1606106680 >> [v2:172.21.15.19:6808/86194,v1:172.21.15.19:6809/86194] conn(0x7ffb640725e0 0x7ffb640749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7f9e54ff9640 1 --2- 172.21.15.19:0/1227251426 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9e4406c1c0 0x7f9e4406e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.395+0000 7f9e57fff640 1 --2- 172.21.15.19:0/1227251426 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9e4406c1c0 0x7f9e4406e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.396+0000 7ffb7ca7f640 10 monclient: get_auth_request con 0x7ffb640725e0 auth_method 0 2023-10-06T13:53:19.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.396+0000 7f9e57fff640 10 monclient: get_auth_request con 0x7f9e4406c1c0 auth_method 0 2023-10-06T13:53:19.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.396+0000 7f9e57fff640 1 --2- 172.21.15.19:0/1227251426 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9e4406c1c0 0x7f9e4406e670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c003cc0 tx=0x7f9e4c0079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.396+0000 7f9e54ff9640 10 client.4244.objecter ms_handle_connect 0x7f9e4406c1c0 2023-10-06T13:53:19.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.401+0000 7ffb74ff9640 10 client.4241.objecter ms_handle_connect 0x7ffb640725e0 2023-10-06T13:53:19.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.401+0000 7ffb7ca7f640 1 --2- 172.21.15.19:0/1606106680 >> [v2:172.21.15.19:6808/86194,v1:172.21.15.19:6809/86194] conn(0x7ffb640725e0 0x7ffb640749f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.501 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.402+0000 7f9e54ff9640 1 -- 172.21.15.19:0/1227251426 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9e4807ad10 con 0x7f9e50031580 2023-10-06T13:53:19.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.402+0000 7f9e54ff9640 10 client.4244.objecter ms_dispatch 0x7f9e58000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:19.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.402+0000 7f9e54ff9640 3 client.4244.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:19.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.402+0000 7f9e54ff9640 3 client.4244.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:19.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.402+0000 7f9e54ff9640 10 client.4244.objecter checking command 1 2023-10-06T13:53:19.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7f9e54ff9640 1 --2- 172.21.15.19:0/1227251426 >> [v2:172.21.15.83:6810/86125,v1:172.21.15.83:6811/86125] conn(0x7f9e44072590 0x7f9e440749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7f9e54ff9640 20 client.4244.objecter _get_session s=0x7f9e44071ec0 osd=7 3 2023-10-06T13:53:19.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7f9e54ff9640 20 client.4244.objecter put_session s=0x7f9e44071ec0 osd=7 3 2023-10-06T13:53:19.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7f9e54ff9640 15 client.4244.objecter _session_command_op_remove -1 1 2023-10-06T13:53:19.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7f9e54ff9640 20 client.4244.objecter _get_session s=0x7f9e44071ec0 osd=7 3 2023-10-06T13:53:19.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7f9e54ff9640 20 client.4244.objecter get_session s=0x7f9e44071ec0 osd=7 3 2023-10-06T13:53:19.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7f9e54ff9640 15 client.4244.objecter _session_command_op_assign 7 1 2023-10-06T13:53:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7f9e54ff9640 20 client.4244.objecter put_session s=0x7f9e44071ec0 osd=7 4 2023-10-06T13:53:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7fa790ff9640 1 -- 172.21.15.19:0/3323333400 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa78c01d3b0 con 0x7fa7940b3d60 2023-10-06T13:53:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7f9e54ff9640 10 client.4244.objecter _send_command 1 2023-10-06T13:53:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7f9e54ff9640 1 -- 172.21.15.19:0/1227251426 --> [v2:172.21.15.83:6810/86125,v1:172.21.15.83:6811/86125] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e44075050 con 0x7f9e44072590 2023-10-06T13:53:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7f9e54ff9640 20 client.4244.objecter dump_active .. 0 homeless 2023-10-06T13:53:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7f9e54ff9640 1 -- 172.21.15.19:0/1227251426 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e4802f020 con 0x7f9e50031580 2023-10-06T13:53:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7f9e54ff9640 10 monclient: handle_get_version_reply finishing 1 version 23 2023-10-06T13:53:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7fa790ff9640 1 --2- 172.21.15.19:0/3323333400 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa77406c210 0x7fa77406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7f9e5c992640 1 --2- 172.21.15.19:0/1227251426 >> [v2:172.21.15.83:6810/86125,v1:172.21.15.83:6811/86125] conn(0x7f9e44072590 0x7f9e440749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.505 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7fa7937fe640 1 --2- 172.21.15.19:0/3323333400 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa77406c210 0x7fa77406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.505 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.403+0000 7fa7937fe640 10 monclient: get_auth_request con 0x7fa77406c210 auth_method 0 2023-10-06T13:53:19.505 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.404+0000 7f9e5c992640 10 monclient: get_auth_request con 0x7f9e44072590 auth_method 0 2023-10-06T13:53:19.506 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.404+0000 7fa790ff9640 10 client.4315.objecter ms_handle_connect 0x7fa77406c210 2023-10-06T13:53:19.506 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.404+0000 7fa7937fe640 1 --2- 172.21.15.19:0/3323333400 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa77406c210 0x7fa77406e6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa784036000 tx=0x7fa784030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.506 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.404+0000 7fa790ff9640 1 -- 172.21.15.19:0/3323333400 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa78c058cc0 con 0x7fa7940b3d60 2023-10-06T13:53:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.404+0000 7fa790ff9640 10 client.4315.objecter ms_dispatch 0x7fa794000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.404+0000 7fa790ff9640 3 client.4315.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.404+0000 7fa790ff9640 3 client.4315.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa790ff9640 10 client.4315.objecter checking command 1 2023-10-06T13:53:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa790ff9640 1 --2- 172.21.15.19:0/3323333400 >> [v2:172.21.15.19:6817/86193,v1:172.21.15.19:6819/86193] conn(0x7fa7740725e0 0x7fa7740749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa790ff9640 20 client.4315.objecter _get_session s=0x7fa774071f10 osd=1 3 2023-10-06T13:53:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa790ff9640 20 client.4315.objecter put_session s=0x7fa774071f10 osd=1 3 2023-10-06T13:53:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa790ff9640 15 client.4315.objecter _session_command_op_remove -1 1 2023-10-06T13:53:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa790ff9640 20 client.4315.objecter _get_session s=0x7fa774071f10 osd=1 3 2023-10-06T13:53:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa790ff9640 20 client.4315.objecter get_session s=0x7fa774071f10 osd=1 3 2023-10-06T13:53:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa790ff9640 15 client.4315.objecter _session_command_op_assign 1 1 2023-10-06T13:53:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa790ff9640 20 client.4315.objecter put_session s=0x7fa774071f10 osd=1 4 2023-10-06T13:53:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa790ff9640 10 client.4315.objecter _send_command 1 2023-10-06T13:53:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa790ff9640 1 -- 172.21.15.19:0/3323333400 --> [v2:172.21.15.19:6817/86193,v1:172.21.15.19:6819/86193] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7740750a0 con 0x7fa7740725e0 2023-10-06T13:53:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa790ff9640 20 client.4315.objecter dump_active .. 0 homeless 2023-10-06T13:53:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa790ff9640 1 -- 172.21.15.19:0/3323333400 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa78c00ce90 con 0x7fa7940b3d60 2023-10-06T13:53:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa790ff9640 10 monclient: handle_get_version_reply finishing 1 version 23 2023-10-06T13:53:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.405+0000 7fa78bfff640 1 --2- 172.21.15.19:0/3323333400 >> [v2:172.21.15.19:6817/86193,v1:172.21.15.19:6819/86193] conn(0x7fa7740725e0 0x7fa7740749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.406+0000 7fa78bfff640 10 monclient: get_auth_request con 0x7fa7740725e0 auth_method 0 2023-10-06T13:53:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.406+0000 7faae5401640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faae00b37e0 0x7faae01dd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.406+0000 7faae5401640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faae00b37e0 0x7faae01dd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:41304/0 (socket says 172.21.15.19:41304) 2023-10-06T13:53:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.406+0000 7faae5401640 1 -- 172.21.15.19:0/170907513 learned_addr learned my addr 172.21.15.19:0/170907513 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.406+0000 7faae5401640 10 monclient(hunting): get_auth_request con 0x7faae00b37e0 auth_method 0 2023-10-06T13:53:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.406+0000 7faae5401640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.406+0000 7faae5401640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.406+0000 7faae5401640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.407+0000 7faae5401640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.407+0000 7faae5401640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.407+0000 7faae5401640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.408+0000 7f9e5c992640 1 --2- 172.21.15.19:0/1227251426 >> [v2:172.21.15.83:6810/86125,v1:172.21.15.83:6811/86125] conn(0x7f9e44072590 0x7f9e440749a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.408+0000 7f9e54ff9640 10 client.4244.objecter ms_handle_connect 0x7f9e44072590 2023-10-06T13:53:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.410+0000 7fa78bfff640 1 --2- 172.21.15.19:0/3323333400 >> [v2:172.21.15.19:6817/86193,v1:172.21.15.19:6819/86193] conn(0x7fa7740725e0 0x7fa7740749f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.410+0000 7fa790ff9640 10 client.4315.objecter ms_handle_connect 0x7fa7740725e0 2023-10-06T13:53:19.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.410+0000 7faae5401640 10 monclient(hunting): handle_auth_done global_id 4250 payload 824 2023-10-06T13:53:19.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.410+0000 7faae5401640 1 -- 172.21.15.19:0/170907513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae0145430 msgr2=0x7faae01cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.410+0000 7faae5401640 1 --2- 172.21.15.19:0/170907513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae0145430 0x7faae01cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.410+0000 7faae5401640 1 -- 172.21.15.19:0/170907513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae00b3410 msgr2=0x7faae01434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.410+0000 7faae5401640 1 --2- 172.21.15.19:0/170907513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae00b3410 0x7faae01434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.410+0000 7faae5401640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.410+0000 7faae5401640 1 monclient: found mon.noname-c 2023-10-06T13:53:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.410+0000 7faae5401640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.410+0000 7faae5401640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.410+0000 7faae5401640 1 -- 172.21.15.19:0/170907513 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad8009b30 con 0x7faae00b37e0 2023-10-06T13:53:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.411+0000 7fbdfaffd640 1 -- 172.21.15.19:0/643282779 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fbdfc03eae0 con 0x7fbe0413da90 2023-10-06T13:53:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.411+0000 7fbdfaffd640 1 --2- 172.21.15.19:0/643282779 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbde806c210 0x7fbde806e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.513 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.411+0000 7fbe0a4ef640 1 --2- 172.21.15.19:0/643282779 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbde806c210 0x7fbde806e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.513 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.411+0000 7fbe0a4ef640 10 monclient: get_auth_request con 0x7fbde806c210 auth_method 0 2023-10-06T13:53:19.513 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.412+0000 7fbe0a4ef640 1 --2- 172.21.15.19:0/643282779 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbde806c210 0x7fbde806e6c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009990 tx=0x7fbe00009110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.513 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.412+0000 7fbdfaffd640 10 client.4278.objecter ms_handle_connect 0x7fbde806c210 2023-10-06T13:53:19.514 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.414+0000 7faae5401640 1 --2- 172.21.15.19:0/170907513 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faae00b37e0 0x7faae01dd730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faad000c410 tx=0x7faad000c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.514 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.414+0000 7faad67fc640 10 client.?.objecter ms_handle_connect 0x7faae00b37e0 2023-10-06T13:53:19.514 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.414+0000 7faad67fc640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.514 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faad67fc640 1 -- 172.21.15.19:0/170907513 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad00035f0 con 0x7faae00b37e0 2023-10-06T13:53:19.514 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faad67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.514 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faad67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:19.514 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faad67fc640 10 monclient: dump: 2023-10-06T13:53:19.515 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.515 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.515 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.515 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.515 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.515 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.515 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.515 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.515 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.515 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.515 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faad67fc640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.515 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faad67fc640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faad67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faad67fc640 1 -- 172.21.15.19:0/170907513 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad0003770 con 0x7faae00b37e0 2023-10-06T13:53:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faad67fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faad67fc640 1 -- 172.21.15.19:0/170907513 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0003b20 con 0x7faae00b37e0 2023-10-06T13:53:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faad67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faad67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faad67fc640 10 monclient: dump: 2023-10-06T13:53:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.517 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faae7e66640 5 monclient: authenticate success, global_id 4250 2023-10-06T13:53:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faae7e66640 10 monclient: _renew_subs 2023-10-06T13:53:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faad7fff640 4 set_mon_vals no callback set 2023-10-06T13:53:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faae7e66640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faae7e66640 1 -- 172.21.15.19:0/170907513 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae01ddd90 con 0x7faae00b37e0 2023-10-06T13:53:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faae7e66640 10 client.4250.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faae7e66640 10 monclient: _renew_subs 2023-10-06T13:53:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faae7e66640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faae7e66640 1 -- 172.21.15.19:0/170907513 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae01de250 con 0x7faae00b37e0 2023-10-06T13:53:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.418+0000 7faae7e66640 1 librados: init done 2023-10-06T13:53:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.419+0000 7faae7e66640 10 client.4250.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.419+0000 7faae7e66640 15 client.4250.objecter _session_command_op_assign -1 1 2023-10-06T13:53:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.419+0000 7faae7e66640 20 client.4250.objecter _get_session osd=-1 returning homeless 2023-10-06T13:53:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.419+0000 7faae7e66640 10 client.4250.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.419+0000 7faae7e66640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.419+0000 7faae7e66640 1 -- 172.21.15.19:0/170907513 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faae00af380 con 0x7faae00b37e0 2023-10-06T13:53:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.428+0000 7ff7be7fc640 1 -- 172.21.15.19:0/1898729719 <== osd.4 v2:172.21.15.83:6826/86123 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff7ac0750a0 con 0x7ff7ac0725e0 2023-10-06T13:53:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.428+0000 7ff7be7fc640 10 client.4318.objecter ms_dispatch 0x7ff7c8000b90 command_reply(tid 1: 0 ) v1 2023-10-06T13:53:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.428+0000 7ff7be7fc640 10 client.4318.objecter _finish_command 1 = system:0 2023-10-06T13:53:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.428+0000 7ff7be7fc640 20 client.4318.objecter put_session s=0x7ff7ac071f10 osd=4 4 2023-10-06T13:53:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.428+0000 7ff7be7fc640 15 client.4318.objecter _session_command_op_remove 4 1 2023-10-06T13:53:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 1 -- 172.21.15.19:0/643282779 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbdfc050e70 con 0x7fbe0413da90 2023-10-06T13:53:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 10 client.4278.objecter ms_dispatch 0x7fbe04000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 3 client.4278.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 3 client.4278.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 10 client.4278.objecter checking command 1 2023-10-06T13:53:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 1 --2- 172.21.15.19:0/643282779 >> [v2:172.21.15.83:6818/86126,v1:172.21.15.83:6819/86126] conn(0x7fbde80725e0 0x7fbde80749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 20 client.4278.objecter _get_session s=0x7fbde8071f10 osd=5 3 2023-10-06T13:53:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 20 client.4278.objecter put_session s=0x7fbde8071f10 osd=5 3 2023-10-06T13:53:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 15 client.4278.objecter _session_command_op_remove -1 1 2023-10-06T13:53:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 20 client.4278.objecter _get_session s=0x7fbde8071f10 osd=5 3 2023-10-06T13:53:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 20 client.4278.objecter get_session s=0x7fbde8071f10 osd=5 3 2023-10-06T13:53:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 15 client.4278.objecter _session_command_op_assign 5 1 2023-10-06T13:53:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 20 client.4278.objecter put_session s=0x7fbde8071f10 osd=5 4 2023-10-06T13:53:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 10 client.4278.objecter _send_command 1 2023-10-06T13:53:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 1 -- 172.21.15.19:0/643282779 --> [v2:172.21.15.83:6818/86126,v1:172.21.15.83:6819/86126] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbde80750a0 con 0x7fbde80725e0 2023-10-06T13:53:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbdfaffd640 20 client.4278.objecter dump_active .. 0 homeless 2023-10-06T13:53:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fa058ff9640 1 -- 172.21.15.19:0/3510967138 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa05004c3b0 con 0x7fa05c13e2b0 2023-10-06T13:53:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.451+0000 7fbe0acf0640 1 --2- 172.21.15.19:0/643282779 >> [v2:172.21.15.83:6818/86126,v1:172.21.15.83:6819/86126] conn(0x7fbde80725e0 0x7fbde80749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.452+0000 7fbe0acf0640 10 monclient: get_auth_request con 0x7fbde80725e0 auth_method 0 2023-10-06T13:53:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.452+0000 7fa058ff9640 1 --2- 172.21.15.19:0/3510967138 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa04806c210 0x7fa04806e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.452+0000 7fa05bfff640 1 --2- 172.21.15.19:0/3510967138 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa04806c210 0x7fa04806e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.452+0000 7fa05bfff640 10 monclient: get_auth_request con 0x7fa04806c210 auth_method 0 2023-10-06T13:53:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.453+0000 7fa05bfff640 1 --2- 172.21.15.19:0/3510967138 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa04806c210 0x7fa04806e6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa04c000f60 tx=0x7fa04c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.453+0000 7fa058ff9640 10 client.4284.objecter ms_handle_connect 0x7fa04806c210 2023-10-06T13:53:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.454+0000 7faad67fc640 1 -- 172.21.15.19:0/170907513 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7faad0003c80 con 0x7faae00b37e0 2023-10-06T13:53:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.454+0000 7faad67fc640 1 --2- 172.21.15.19:0/170907513 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7faac406c240 0x7faac406e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.455+0000 7faae5c02640 1 --2- 172.21.15.19:0/170907513 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7faac406c240 0x7faac406e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.455+0000 7faae5c02640 10 monclient: get_auth_request con 0x7faac406c240 auth_method 0 2023-10-06T13:53:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7faad67fc640 10 client.4250.objecter ms_handle_connect 0x7faac406c240 2023-10-06T13:53:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7faae5c02640 1 --2- 172.21.15.19:0/170907513 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7faac406c240 0x7faac406e6f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faadc009710 tx=0x7faadc006c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fbdfaffd640 10 client.4278.objecter ms_handle_connect 0x7fbde80725e0 2023-10-06T13:53:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fbe0acf0640 1 --2- 172.21.15.19:0/643282779 >> [v2:172.21.15.83:6818/86126,v1:172.21.15.83:6819/86126] conn(0x7fbde80725e0 0x7fbde80749f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 1 -- 172.21.15.19:0/3510967138 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa05007a3f0 con 0x7fa05c13e2b0 2023-10-06T13:53:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 10 client.4284.objecter ms_dispatch 0x7fa05c000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 3 client.4284.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 3 client.4284.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 10 client.4284.objecter checking command 1 2023-10-06T13:53:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 1 --2- 172.21.15.19:0/3510967138 >> [v2:172.21.15.19:6816/86195,v1:172.21.15.19:6818/86195] conn(0x7fa0480725e0 0x7fa0480749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 20 client.4284.objecter _get_session s=0x7fa048071f10 osd=0 3 2023-10-06T13:53:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 20 client.4284.objecter put_session s=0x7fa048071f10 osd=0 3 2023-10-06T13:53:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 15 client.4284.objecter _session_command_op_remove -1 1 2023-10-06T13:53:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 20 client.4284.objecter _get_session s=0x7fa048071f10 osd=0 3 2023-10-06T13:53:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 20 client.4284.objecter get_session s=0x7fa048071f10 osd=0 3 2023-10-06T13:53:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 15 client.4284.objecter _session_command_op_assign 0 1 2023-10-06T13:53:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 20 client.4284.objecter put_session s=0x7fa048071f10 osd=0 4 2023-10-06T13:53:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 10 client.4284.objecter _send_command 1 2023-10-06T13:53:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 1 -- 172.21.15.19:0/3510967138 --> [v2:172.21.15.19:6816/86195,v1:172.21.15.19:6818/86195] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0480750a0 con 0x7fa0480725e0 2023-10-06T13:53:19.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 20 client.4284.objecter dump_active .. 0 homeless 2023-10-06T13:53:19.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 1 -- 172.21.15.19:0/3510967138 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa050076730 con 0x7fa05c13e2b0 2023-10-06T13:53:19.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.456+0000 7fa058ff9640 10 monclient: handle_get_version_reply finishing 1 version 23 2023-10-06T13:53:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.458+0000 7ff7d000b640 10 client.4318.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-10-06T13:53:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.458+0000 7ff7d000b640 15 client.4318.objecter _session_command_op_assign -1 2 2023-10-06T13:53:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.458+0000 7ff7d000b640 20 client.4318.objecter _get_session s=0x7ff7ac071f10 osd=4 3 2023-10-06T13:53:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.458+0000 7ff7d000b640 20 client.4318.objecter put_session s=0x7ff7ac071f10 osd=4 3 2023-10-06T13:53:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.458+0000 7ff7d000b640 20 client.4318.objecter _get_session s=0x7ff7ac071f10 osd=4 3 2023-10-06T13:53:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.458+0000 7ff7d000b640 15 client.4318.objecter _session_command_op_remove -1 2 2023-10-06T13:53:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.458+0000 7ff7d000b640 20 client.4318.objecter get_session s=0x7ff7ac071f10 osd=4 3 2023-10-06T13:53:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.458+0000 7ff7d000b640 15 client.4318.objecter _session_command_op_assign 4 2 2023-10-06T13:53:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.458+0000 7ff7d000b640 20 client.4318.objecter put_session s=0x7ff7ac071f10 osd=4 4 2023-10-06T13:53:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.458+0000 7ff7d000b640 10 client.4318.objecter _send_command 2 2023-10-06T13:53:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.458+0000 7ff7d000b640 1 -- 172.21.15.19:0/1898729719 --> [v2:172.21.15.83:6826/86123,v1:172.21.15.83:6827/86123] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff7c0036880 con 0x7ff7ac0725e0 2023-10-06T13:53:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 1 -- 172.21.15.19:0/170907513 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7faad0058320 con 0x7faae00b37e0 2023-10-06T13:53:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 10 client.4250.objecter ms_dispatch 0x7faae0000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 3 client.4250.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 3 client.4250.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 10 client.4250.objecter checking command 1 2023-10-06T13:53:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 1 --2- 172.21.15.19:0/170907513 >> [v2:172.21.15.19:6800/86196,v1:172.21.15.19:6801/86196] conn(0x7faac4072610 0x7faac4074a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 20 client.4250.objecter _get_session s=0x7faac4071f40 osd=3 3 2023-10-06T13:53:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 20 client.4250.objecter put_session s=0x7faac4071f40 osd=3 3 2023-10-06T13:53:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 15 client.4250.objecter _session_command_op_remove -1 1 2023-10-06T13:53:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 20 client.4250.objecter _get_session s=0x7faac4071f40 osd=3 3 2023-10-06T13:53:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 20 client.4250.objecter get_session s=0x7faac4071f40 osd=3 3 2023-10-06T13:53:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 15 client.4250.objecter _session_command_op_assign 3 1 2023-10-06T13:53:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 20 client.4250.objecter put_session s=0x7faac4071f40 osd=3 4 2023-10-06T13:53:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 10 client.4250.objecter _send_command 1 2023-10-06T13:53:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 1 -- 172.21.15.19:0/170907513 --> [v2:172.21.15.19:6800/86196,v1:172.21.15.19:6801/86196] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faac40750d0 con 0x7faac4072610 2023-10-06T13:53:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 20 client.4250.objecter dump_active .. 0 homeless 2023-10-06T13:53:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 1 -- 172.21.15.19:0/170907513 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faad0053f60 con 0x7faae00b37e0 2023-10-06T13:53:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.460+0000 7faad67fc640 10 monclient: handle_get_version_reply finishing 1 version 23 2023-10-06T13:53:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.461+0000 7fa790ff9640 1 -- 172.21.15.19:0/3323333400 <== osd.1 v2:172.21.15.19:6817/86193 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa7740750a0 con 0x7fa7740725e0 2023-10-06T13:53:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.461+0000 7fa790ff9640 10 client.4315.objecter ms_dispatch 0x7fa794000b90 command_reply(tid 1: 0 ) v1 2023-10-06T13:53:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.461+0000 7fa790ff9640 10 client.4315.objecter _finish_command 1 = system:0 2023-10-06T13:53:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.461+0000 7fa790ff9640 20 client.4315.objecter put_session s=0x7fa774071f10 osd=1 4 2023-10-06T13:53:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.461+0000 7fa790ff9640 15 client.4315.objecter _session_command_op_remove 1 1 2023-10-06T13:53:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.461+0000 7faae6403640 1 --2- 172.21.15.19:0/170907513 >> [v2:172.21.15.19:6800/86196,v1:172.21.15.19:6801/86196] conn(0x7faac4072610 0x7faac4074a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.461+0000 7faae6403640 10 monclient: get_auth_request con 0x7faac4072610 auth_method 0 2023-10-06T13:53:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.467+0000 7ff7be7fc640 1 -- 172.21.15.19:0/1898729719 <== osd.4 v2:172.21.15.83:6826/86123 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff7b8009480 con 0x7ff7ac0725e0 2023-10-06T13:53:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.467+0000 7ff7be7fc640 10 client.4318.objecter ms_dispatch 0x7ff7c8000b90 command_reply(tid 2: 0 ) v1 2023-10-06T13:53:19.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.467+0000 7ff7be7fc640 10 client.4318.objecter _finish_command 2 = system:0 2023-10-06T13:53:19.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.467+0000 7ff7be7fc640 20 client.4318.objecter put_session s=0x7ff7ac071f10 osd=4 4 2023-10-06T13:53:19.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.467+0000 7ff7be7fc640 15 client.4318.objecter _session_command_op_remove 4 2 2023-10-06T13:53:19.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.467+0000 7ff7d000b640 10 librados: watch_flush enter 2023-10-06T13:53:19.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7ffa97fff640 1 -- 172.21.15.19:0/2166371806 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ffa90007c90 con 0x7ffaa00b00c0 2023-10-06T13:53:19.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7faae6403640 1 --2- 172.21.15.19:0/170907513 >> [v2:172.21.15.19:6800/86196,v1:172.21.15.19:6801/86196] conn(0x7faac4072610 0x7faac4074a20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7faad67fc640 10 client.4250.objecter ms_handle_connect 0x7faac4072610 2023-10-06T13:53:19.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7ff7d000b640 10 librados: watch_flush exit 2023-10-06T13:53:19.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7ff7d000b640 10 client.4318.objecter close_session for osd.4 2023-10-06T13:53:19.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7ff7d000b640 1 -- 172.21.15.19:0/1898729719 >> [v2:172.21.15.83:6826/86123,v1:172.21.15.83:6827/86123] conn(0x7ff7ac0725e0 msgr2=0x7ff7ac0749f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7ff7d000b640 1 --2- 172.21.15.19:0/1898729719 >> [v2:172.21.15.83:6826/86123,v1:172.21.15.83:6827/86123] conn(0x7ff7ac0725e0 0x7ff7ac0749f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7ff7d000b640 20 client.4318.objecter put_session s=0x7ff7ac071f10 osd=4 1 2023-10-06T13:53:19.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7ff7d000b640 20 client.4318.objecter shutdown clearing up homeless session... 2023-10-06T13:53:19.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7ff7d000b640 10 client.4318.objecter successfully canceled tick 2023-10-06T13:53:19.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7ff7d000b640 1 -- 172.21.15.19:0/1898729719 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ff7ac06c210 msgr2=0x7ff7ac06e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7ff7d000b640 1 --2- 172.21.15.19:0/1898729719 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ff7ac06c210 0x7ff7ac06e6c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4001560 tx=0x7ff7c4007990 comp rx=0 tx=0).stop 2023-10-06T13:53:19.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7ff7d000b640 10 monclient: shutdown 2023-10-06T13:53:19.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7ff7d000b640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7ff7d000b640 1 -- 172.21.15.19:0/1898729719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7c00c67c0 msgr2=0x7ff7c015f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.468+0000 7ff7d000b640 1 --2- 172.21.15.19:0/1898729719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7c00c67c0 0x7ff7c015f970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff7c804f270 tx=0x7ff7c8068560 comp rx=0 tx=0).stop 2023-10-06T13:53:19.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.469+0000 7ffa97fff640 1 --2- 172.21.15.19:0/2166371806 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ffa8006c210 0x7ffa8006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.469+0000 7ff7ce5a8640 1 -- 172.21.15.19:0/1898729719 reap_dead start 2023-10-06T13:53:19.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.469+0000 7ffa967fc640 1 --2- 172.21.15.19:0/2166371806 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ffa8006c210 0x7ffa8006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.469+0000 7ffa967fc640 10 monclient: get_auth_request con 0x7ffa8006c210 auth_method 0 2023-10-06T13:53:19.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.469+0000 7ff7d000b640 1 -- 172.21.15.19:0/1898729719 shutdown_connections 2023-10-06T13:53:19.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.469+0000 7ff7d000b640 1 -- 172.21.15.19:0/1898729719 >> 172.21.15.19:0/1898729719 conn(0x7ff7c0039ea0 msgr2=0x7ff7c00384f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.469+0000 7ff7d000b640 1 -- 172.21.15.19:0/1898729719 shutdown_connections 2023-10-06T13:53:19.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.469+0000 7ff7d000b640 1 -- 172.21.15.19:0/1898729719 wait complete. 2023-10-06T13:53:19.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.469+0000 7ff7d000b640 1 librados: shutdown 2023-10-06T13:53:19.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.469+0000 7ffa97fff640 10 client.4281.objecter ms_handle_connect 0x7ffa8006c210 2023-10-06T13:53:19.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.469+0000 7ffa967fc640 1 --2- 172.21.15.19:0/2166371806 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ffa8006c210 0x7ffa8006e6c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffaa01450a0 tx=0x7ffa88009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.472+0000 7fa060e21640 1 --2- 172.21.15.19:0/3510967138 >> [v2:172.21.15.19:6816/86195,v1:172.21.15.19:6818/86195] conn(0x7fa0480725e0 0x7fa0480749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.472+0000 7ffb74ff9640 1 -- 172.21.15.19:0/1606106680 <== osd.2 v2:172.21.15.19:6808/86194 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ffb640750a0 con 0x7ffb640725e0 2023-10-06T13:53:19.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.472+0000 7ffb74ff9640 10 client.4241.objecter ms_dispatch 0x7ffb78000b90 command_reply(tid 1: 0 ) v1 2023-10-06T13:53:19.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.472+0000 7ffb74ff9640 10 client.4241.objecter _finish_command 1 = system:0 2023-10-06T13:53:19.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.472+0000 7ffb74ff9640 20 client.4241.objecter put_session s=0x7ffb64071f10 osd=2 4 2023-10-06T13:53:19.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.472+0000 7ffb74ff9640 15 client.4241.objecter _session_command_op_remove 2 1 2023-10-06T13:53:19.538 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.476+0000 7fbdfaffd640 1 -- 172.21.15.19:0/643282779 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdfc0776e0 con 0x7fbe0413da90 2023-10-06T13:53:19.538 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.476+0000 7fbdfaffd640 10 monclient: handle_get_version_reply finishing 1 version 23 2023-10-06T13:53:19.538 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.477+0000 7f9e54ff9640 1 -- 172.21.15.19:0/1227251426 <== osd.7 v2:172.21.15.83:6810/86125 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9e44075050 con 0x7f9e44072590 2023-10-06T13:53:19.538 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.477+0000 7f9e54ff9640 10 client.4244.objecter ms_dispatch 0x7f9e58000b90 command_reply(tid 1: 0 ) v1 2023-10-06T13:53:19.538 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.477+0000 7f9e54ff9640 10 client.4244.objecter _finish_command 1 = system:0 2023-10-06T13:53:19.538 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.477+0000 7f9e54ff9640 20 client.4244.objecter put_session s=0x7f9e44071ec0 osd=7 4 2023-10-06T13:53:19.538 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.477+0000 7f9e54ff9640 15 client.4244.objecter _session_command_op_remove 7 1 2023-10-06T13:53:19.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.477+0000 7fa060e21640 10 monclient: get_auth_request con 0x7fa0480725e0 auth_method 0 2023-10-06T13:53:19.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.477+0000 7ffa97fff640 1 -- 172.21.15.19:0/2166371806 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ffa90058b50 con 0x7ffaa00b00c0 2023-10-06T13:53:19.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.477+0000 7ffa97fff640 10 client.4281.objecter ms_dispatch 0x7ffaa0000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:19.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.477+0000 7ffa97fff640 3 client.4281.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:19.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.477+0000 7ffa97fff640 3 client.4281.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:19.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.478+0000 7ffa97fff640 10 client.4281.objecter checking command 1 2023-10-06T13:53:19.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.478+0000 7ffa97fff640 1 --2- 172.21.15.19:0/2166371806 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x7ffa800725e0 0x7ffa800749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.478+0000 7ffa97fff640 20 client.4281.objecter _get_session s=0x7ffa80071f10 osd=6 3 2023-10-06T13:53:19.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.478+0000 7ffa97fff640 20 client.4281.objecter put_session s=0x7ffa80071f10 osd=6 3 2023-10-06T13:53:19.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.478+0000 7ffa97fff640 15 client.4281.objecter _session_command_op_remove -1 1 2023-10-06T13:53:19.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.478+0000 7ffa97fff640 20 client.4281.objecter _get_session s=0x7ffa80071f10 osd=6 3 2023-10-06T13:53:19.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.478+0000 7ffa97fff640 20 client.4281.objecter get_session s=0x7ffa80071f10 osd=6 3 2023-10-06T13:53:19.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.478+0000 7ffa97fff640 15 client.4281.objecter _session_command_op_assign 6 1 2023-10-06T13:53:19.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.478+0000 7ffa97fff640 20 client.4281.objecter put_session s=0x7ffa80071f10 osd=6 4 2023-10-06T13:53:19.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.478+0000 7ffa97fff640 10 client.4281.objecter _send_command 1 2023-10-06T13:53:19.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.478+0000 7ffa97fff640 1 -- 172.21.15.19:0/2166371806 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa800750a0 con 0x7ffa800725e0 2023-10-06T13:53:19.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.478+0000 7ffa97fff640 20 client.4281.objecter dump_active .. 0 homeless 2023-10-06T13:53:19.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.478+0000 7ffa97fff640 1 -- 172.21.15.19:0/2166371806 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa90054790 con 0x7ffaa00b00c0 2023-10-06T13:53:19.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.478+0000 7ffa97fff640 10 monclient: handle_get_version_reply finishing 1 version 23 2023-10-06T13:53:19.541 INFO:teuthology.orchestra.run.smithi019.stdout:64424509446 2023-10-06T13:53:19.541 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T13:53:19.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.488+0000 7ffa9f7fe640 1 --2- 172.21.15.19:0/2166371806 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x7ffa800725e0 0x7ffa800749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.488+0000 7ffa9f7fe640 10 monclient: get_auth_request con 0x7ffa800725e0 auth_method 0 2023-10-06T13:53:19.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.489+0000 7fa799aaf640 10 client.4315.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-10-06T13:53:19.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.489+0000 7fa799aaf640 15 client.4315.objecter _session_command_op_assign -1 2 2023-10-06T13:53:19.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.489+0000 7fa799aaf640 20 client.4315.objecter _get_session s=0x7fa774071f10 osd=1 3 2023-10-06T13:53:19.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.489+0000 7fa799aaf640 20 client.4315.objecter put_session s=0x7fa774071f10 osd=1 3 2023-10-06T13:53:19.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.489+0000 7fa799aaf640 20 client.4315.objecter _get_session s=0x7fa774071f10 osd=1 3 2023-10-06T13:53:19.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.489+0000 7fa799aaf640 15 client.4315.objecter _session_command_op_remove -1 2 2023-10-06T13:53:19.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.489+0000 7fa799aaf640 20 client.4315.objecter get_session s=0x7fa774071f10 osd=1 3 2023-10-06T13:53:19.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.489+0000 7fa799aaf640 15 client.4315.objecter _session_command_op_assign 1 2 2023-10-06T13:53:19.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.489+0000 7fa799aaf640 20 client.4315.objecter put_session s=0x7fa774071f10 osd=1 4 2023-10-06T13:53:19.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.489+0000 7fa799aaf640 10 client.4315.objecter _send_command 2 2023-10-06T13:53:19.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.489+0000 7fa799aaf640 1 -- 172.21.15.19:0/3323333400 --> [v2:172.21.15.19:6817/86193,v1:172.21.15.19:6819/86193] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa7940aee30 con 0x7fa7740725e0 2023-10-06T13:53:19.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.491+0000 7ffa97fff640 10 client.4281.objecter ms_handle_connect 0x7ffa800725e0 2023-10-06T13:53:19.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.491+0000 7ffa9f7fe640 1 --2- 172.21.15.19:0/2166371806 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x7ffa800725e0 0x7ffa800749f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.493+0000 7fa058ff9640 10 client.4284.objecter ms_handle_connect 0x7fa0480725e0 2023-10-06T13:53:19.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.493+0000 7fa060e21640 1 --2- 172.21.15.19:0/3510967138 >> [v2:172.21.15.19:6816/86195,v1:172.21.15.19:6818/86195] conn(0x7fa0480725e0 0x7fa0480749f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7fbdfaffd640 1 -- 172.21.15.19:0/643282779 <== osd.5 v2:172.21.15.83:6818/86126 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbde80750a0 con 0x7fbde80725e0 2023-10-06T13:53:19.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7fbdfaffd640 10 client.4278.objecter ms_dispatch 0x7fbe04000b90 command_reply(tid 1: 0 ) v1 2023-10-06T13:53:19.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7fbdfaffd640 10 client.4278.objecter _finish_command 1 = system:0 2023-10-06T13:53:19.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7fbdfaffd640 20 client.4278.objecter put_session s=0x7fbde8071f10 osd=5 4 2023-10-06T13:53:19.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7fbdfaffd640 15 client.4278.objecter _session_command_op_remove 5 1 2023-10-06T13:53:19.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7ffb7d280640 10 client.4241.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-10-06T13:53:19.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7ffb7d280640 15 client.4241.objecter _session_command_op_assign -1 2 2023-10-06T13:53:19.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7ffb7d280640 20 client.4241.objecter _get_session s=0x7ffb64071f10 osd=2 3 2023-10-06T13:53:19.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7ffb7d280640 20 client.4241.objecter put_session s=0x7ffb64071f10 osd=2 3 2023-10-06T13:53:19.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7ffb7d280640 20 client.4241.objecter _get_session s=0x7ffb64071f10 osd=2 3 2023-10-06T13:53:19.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7ffb7d280640 15 client.4241.objecter _session_command_op_remove -1 2 2023-10-06T13:53:19.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7ffb7d280640 20 client.4241.objecter get_session s=0x7ffb64071f10 osd=2 3 2023-10-06T13:53:19.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7ffb7d280640 15 client.4241.objecter _session_command_op_assign 2 2 2023-10-06T13:53:19.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7ffb7d280640 20 client.4241.objecter put_session s=0x7ffb64071f10 osd=2 4 2023-10-06T13:53:19.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7ffb7d280640 10 client.4241.objecter _send_command 2 2023-10-06T13:53:19.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.500+0000 7ffb7d280640 1 -- 172.21.15.19:0/1606106680 --> [v2:172.21.15.19:6808/86194,v1:172.21.15.19:6809/86194] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffb780751e0 con 0x7ffb640725e0 2023-10-06T13:53:19.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.501+0000 7f9e5e3f5640 10 client.4244.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-10-06T13:53:19.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.501+0000 7f9e5e3f5640 15 client.4244.objecter _session_command_op_assign -1 2 2023-10-06T13:53:19.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.501+0000 7f9e5e3f5640 20 client.4244.objecter _get_session s=0x7f9e44071ec0 osd=7 3 2023-10-06T13:53:19.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.501+0000 7f9e5e3f5640 20 client.4244.objecter put_session s=0x7f9e44071ec0 osd=7 3 2023-10-06T13:53:19.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.501+0000 7f9e5e3f5640 20 client.4244.objecter _get_session s=0x7f9e44071ec0 osd=7 3 2023-10-06T13:53:19.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.501+0000 7f9e5e3f5640 15 client.4244.objecter _session_command_op_remove -1 2 2023-10-06T13:53:19.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.501+0000 7f9e5e3f5640 20 client.4244.objecter get_session s=0x7f9e44071ec0 osd=7 3 2023-10-06T13:53:19.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.501+0000 7f9e5e3f5640 15 client.4244.objecter _session_command_op_assign 7 2 2023-10-06T13:53:19.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.501+0000 7f9e5e3f5640 20 client.4244.objecter put_session s=0x7f9e44071ec0 osd=7 4 2023-10-06T13:53:19.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.501+0000 7f9e5e3f5640 10 client.4244.objecter _send_command 2 2023-10-06T13:53:19.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.501+0000 7f9e5e3f5640 1 -- 172.21.15.19:0/1227251426 --> [v2:172.21.15.83:6810/86125,v1:172.21.15.83:6811/86125] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9e50032820 con 0x7f9e44072590 2023-10-06T13:53:19.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.506+0000 7fa790ff9640 1 -- 172.21.15.19:0/3323333400 <== osd.1 v2:172.21.15.19:6817/86193 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa77c009280 con 0x7fa7740725e0 2023-10-06T13:53:19.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.506+0000 7fa790ff9640 10 client.4315.objecter ms_dispatch 0x7fa794000b90 command_reply(tid 2: 0 ) v1 2023-10-06T13:53:19.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.506+0000 7fa790ff9640 10 client.4315.objecter _finish_command 2 = system:0 2023-10-06T13:53:19.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.506+0000 7fa790ff9640 20 client.4315.objecter put_session s=0x7fa774071f10 osd=1 4 2023-10-06T13:53:19.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.506+0000 7fa790ff9640 15 client.4315.objecter _session_command_op_remove 1 2 2023-10-06T13:53:19.549 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 10 librados: watch_flush enter 2023-10-06T13:53:19.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 10 librados: watch_flush exit 2023-10-06T13:53:19.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 10 client.4315.objecter close_session for osd.1 2023-10-06T13:53:19.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 1 -- 172.21.15.19:0/3323333400 >> [v2:172.21.15.19:6817/86193,v1:172.21.15.19:6819/86193] conn(0x7fa7740725e0 msgr2=0x7fa7740749f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 1 --2- 172.21.15.19:0/3323333400 >> [v2:172.21.15.19:6817/86193,v1:172.21.15.19:6819/86193] conn(0x7fa7740725e0 0x7fa7740749f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 20 client.4315.objecter put_session s=0x7fa774071f10 osd=1 1 2023-10-06T13:53:19.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 20 client.4315.objecter shutdown clearing up homeless session... 2023-10-06T13:53:19.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 10 client.4315.objecter successfully canceled tick 2023-10-06T13:53:19.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 1 -- 172.21.15.19:0/3323333400 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa77406c210 msgr2=0x7fa77406e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 1 --2- 172.21.15.19:0/3323333400 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa77406c210 0x7fa77406e6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa784036000 tx=0x7fa784030000 comp rx=0 tx=0).stop 2023-10-06T13:53:19.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 10 monclient: shutdown 2023-10-06T13:53:19.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 1 -- 172.21.15.19:0/3323333400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7940b3d60 msgr2=0x7fa79413da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 1 --2- 172.21.15.19:0/3323333400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7940b3d60 0x7fa79413da30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00b9a0 tx=0x7fa78c00be60 comp rx=0 tx=0).stop 2023-10-06T13:53:19.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa793fff640 1 -- 172.21.15.19:0/3323333400 reap_dead start 2023-10-06T13:53:19.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 1 -- 172.21.15.19:0/3323333400 shutdown_connections 2023-10-06T13:53:19.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.507+0000 7fa799aaf640 1 -- 172.21.15.19:0/3323333400 >> 172.21.15.19:0/3323333400 conn(0x7fa7940b6e00 msgr2=0x7fa79406b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.508+0000 7fa799aaf640 1 -- 172.21.15.19:0/3323333400 shutdown_connections 2023-10-06T13:53:19.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.508+0000 7fa799aaf640 1 -- 172.21.15.19:0/3323333400 wait complete. 2023-10-06T13:53:19.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.508+0000 7fa799aaf640 1 librados: shutdown 2023-10-06T13:53:19.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.510+0000 7ffb74ff9640 1 -- 172.21.15.19:0/1606106680 <== osd.2 v2:172.21.15.19:6808/86194 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffb780751e0 con 0x7ffb640725e0 2023-10-06T13:53:19.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.510+0000 7ffb74ff9640 10 client.4241.objecter ms_dispatch 0x7ffb78000b90 command_reply(tid 2: 0 ) v1 2023-10-06T13:53:19.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.510+0000 7ffb74ff9640 10 client.4241.objecter _finish_command 2 = system:0 2023-10-06T13:53:19.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.510+0000 7ffb74ff9640 20 client.4241.objecter put_session s=0x7ffb64071f10 osd=2 4 2023-10-06T13:53:19.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.510+0000 7ffb74ff9640 15 client.4241.objecter _session_command_op_remove 2 2 2023-10-06T13:53:19.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.510+0000 7ffb7d280640 10 librados: watch_flush enter 2023-10-06T13:53:19.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.510+0000 7ffb7d280640 10 librados: watch_flush exit 2023-10-06T13:53:19.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.510+0000 7ffb7d280640 10 client.4241.objecter close_session for osd.2 2023-10-06T13:53:19.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.510+0000 7ffb7d280640 1 -- 172.21.15.19:0/1606106680 >> [v2:172.21.15.19:6808/86194,v1:172.21.15.19:6809/86194] conn(0x7ffb640725e0 msgr2=0x7ffb640749f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.510+0000 7ffb7d280640 1 --2- 172.21.15.19:0/1606106680 >> [v2:172.21.15.19:6808/86194,v1:172.21.15.19:6809/86194] conn(0x7ffb640725e0 0x7ffb640749f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.511+0000 7ffb7d280640 20 client.4241.objecter put_session s=0x7ffb64071f10 osd=2 1 2023-10-06T13:53:19.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.511+0000 7ffb7d280640 20 client.4241.objecter shutdown clearing up homeless session... 2023-10-06T13:53:19.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.511+0000 7ffb7d280640 10 client.4241.objecter successfully canceled tick 2023-10-06T13:53:19.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.512+0000 7ffb7d280640 1 -- 172.21.15.19:0/1606106680 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ffb6406c210 msgr2=0x7ffb6406e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.512+0000 7ffb7d280640 1 --2- 172.21.15.19:0/1606106680 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ffb6406c210 0x7ffb6406e6c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffb700070b0 tx=0x7ffb70007040 comp rx=0 tx=0).stop 2023-10-06T13:53:19.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.512+0000 7ffb7d280640 10 monclient: shutdown 2023-10-06T13:53:19.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.512+0000 7ffb7d280640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.512+0000 7ffb7d280640 1 -- 172.21.15.19:0/1606106680 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffb7806da00 msgr2=0x7ffb78137850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.512+0000 7ffb7d280640 1 --2- 172.21.15.19:0/1606106680 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ffb7806da00 0x7ffb78137850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00c970 tx=0x7ffb6c004730 comp rx=0 tx=0).stop 2023-10-06T13:53:19.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.512+0000 7ffb7ca7f640 1 -- 172.21.15.19:0/1606106680 reap_dead start 2023-10-06T13:53:19.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.513+0000 7ffb7d280640 1 -- 172.21.15.19:0/1606106680 shutdown_connections 2023-10-06T13:53:19.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.513+0000 7ffb7d280640 1 -- 172.21.15.19:0/1606106680 >> 172.21.15.19:0/1606106680 conn(0x7ffb7806af30 msgr2=0x7ffb780704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.513+0000 7ffb7d280640 1 -- 172.21.15.19:0/1606106680 shutdown_connections 2023-10-06T13:53:19.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.513+0000 7ffb7d280640 1 -- 172.21.15.19:0/1606106680 wait complete. 2023-10-06T13:53:19.555 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.513+0000 7ffb7d280640 1 librados: shutdown 2023-10-06T13:53:19.555 INFO:teuthology.orchestra.run.smithi019.stdout:68719476743 2023-10-06T13:53:19.556 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T13:53:19.556 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.526+0000 7f9e54ff9640 1 -- 172.21.15.19:0/1227251426 <== osd.7 v2:172.21.15.83:6810/86125 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9e5804e570 con 0x7f9e44072590 2023-10-06T13:53:19.556 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.526+0000 7f9e54ff9640 10 client.4244.objecter ms_dispatch 0x7f9e58000b90 command_reply(tid 2: 0 ) v1 2023-10-06T13:53:19.556 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.526+0000 7f9e54ff9640 10 client.4244.objecter _finish_command 2 = system:0 2023-10-06T13:53:19.556 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.526+0000 7f9e54ff9640 20 client.4244.objecter put_session s=0x7f9e44071ec0 osd=7 4 2023-10-06T13:53:19.556 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.526+0000 7f9e54ff9640 15 client.4244.objecter _session_command_op_remove 7 2 2023-10-06T13:53:19.557 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 10 librados: watch_flush enter 2023-10-06T13:53:19.557 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 10 librados: watch_flush exit 2023-10-06T13:53:19.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 10 client.4244.objecter close_session for osd.7 2023-10-06T13:53:19.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 1 -- 172.21.15.19:0/1227251426 >> [v2:172.21.15.83:6810/86125,v1:172.21.15.83:6811/86125] conn(0x7f9e44072590 msgr2=0x7f9e440749a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 1 --2- 172.21.15.19:0/1227251426 >> [v2:172.21.15.83:6810/86125,v1:172.21.15.83:6811/86125] conn(0x7f9e44072590 0x7f9e440749a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 20 client.4244.objecter put_session s=0x7f9e44071ec0 osd=7 1 2023-10-06T13:53:19.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 20 client.4244.objecter shutdown clearing up homeless session... 2023-10-06T13:53:19.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 10 client.4244.objecter successfully canceled tick 2023-10-06T13:53:19.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 1 -- 172.21.15.19:0/1227251426 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9e4406c1c0 msgr2=0x7f9e4406e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 1 --2- 172.21.15.19:0/1227251426 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9e4406c1c0 0x7f9e4406e670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c003cc0 tx=0x7f9e4c0079b0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 10 monclient: shutdown 2023-10-06T13:53:19.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 1 -- 172.21.15.19:0/1227251426 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9e50031580 msgr2=0x7f9e500cebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 1 --2- 172.21.15.19:0/1227251426 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9e50031580 0x7f9e500cebc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9e480098c0 tx=0x7f9e480046c0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e5c992640 1 -- 172.21.15.19:0/1227251426 reap_dead start 2023-10-06T13:53:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 1 -- 172.21.15.19:0/1227251426 shutdown_connections 2023-10-06T13:53:19.559 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.527+0000 7f9e3e7fc640 1 -- 172.21.15.19:0/1227251426 >> 172.21.15.19:0/1227251426 conn(0x7f9e50035e70 msgr2=0x7f9e50006440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.559 INFO:teuthology.orchestra.run.smithi019.stdout:64424509444 2023-10-06T13:53:19.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T13:53:19.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.528+0000 7f9e3e7fc640 1 -- 172.21.15.19:0/1227251426 shutdown_connections 2023-10-06T13:53:19.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.528+0000 7f9e3e7fc640 1 -- 172.21.15.19:0/1227251426 wait complete. 2023-10-06T13:53:19.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.528+0000 7f9e3e7fc640 1 librados: shutdown 2023-10-06T13:53:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.531+0000 7fbe0c753640 10 client.4278.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-10-06T13:53:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.531+0000 7fbe0c753640 15 client.4278.objecter _session_command_op_assign -1 2 2023-10-06T13:53:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.531+0000 7fbe0c753640 20 client.4278.objecter _get_session s=0x7fbde8071f10 osd=5 3 2023-10-06T13:53:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.531+0000 7fbe0c753640 20 client.4278.objecter put_session s=0x7fbde8071f10 osd=5 3 2023-10-06T13:53:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.531+0000 7fbe0c753640 20 client.4278.objecter _get_session s=0x7fbde8071f10 osd=5 3 2023-10-06T13:53:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.531+0000 7fbe0c753640 15 client.4278.objecter _session_command_op_remove -1 2 2023-10-06T13:53:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.531+0000 7fbe0c753640 20 client.4278.objecter get_session s=0x7fbde8071f10 osd=5 3 2023-10-06T13:53:19.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.531+0000 7fbe0c753640 15 client.4278.objecter _session_command_op_assign 5 2 2023-10-06T13:53:19.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.531+0000 7fbe0c753640 20 client.4278.objecter put_session s=0x7fbde8071f10 osd=5 4 2023-10-06T13:53:19.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.531+0000 7fbe0c753640 10 client.4278.objecter _send_command 2 2023-10-06T13:53:19.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.531+0000 7fbe0c753640 1 -- 172.21.15.19:0/643282779 --> [v2:172.21.15.83:6818/86126,v1:172.21.15.83:6819/86126] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe040a9b90 con 0x7fbde80725e0 2023-10-06T13:53:19.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbdfaffd640 1 -- 172.21.15.19:0/643282779 <== osd.5 v2:172.21.15.83:6818/86126 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbdf400a2f0 con 0x7fbde80725e0 2023-10-06T13:53:19.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbdfaffd640 10 client.4278.objecter ms_dispatch 0x7fbe04000b90 command_reply(tid 2: 0 ) v1 2023-10-06T13:53:19.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbdfaffd640 10 client.4278.objecter _finish_command 2 = system:0 2023-10-06T13:53:19.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbdfaffd640 20 client.4278.objecter put_session s=0x7fbde8071f10 osd=5 4 2023-10-06T13:53:19.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbdfaffd640 15 client.4278.objecter _session_command_op_remove 5 2 2023-10-06T13:53:19.563 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 10 librados: watch_flush enter 2023-10-06T13:53:19.563 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 10 librados: watch_flush exit 2023-10-06T13:53:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 10 client.4278.objecter close_session for osd.5 2023-10-06T13:53:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 1 -- 172.21.15.19:0/643282779 >> [v2:172.21.15.83:6818/86126,v1:172.21.15.83:6819/86126] conn(0x7fbde80725e0 msgr2=0x7fbde80749f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 1 --2- 172.21.15.19:0/643282779 >> [v2:172.21.15.83:6818/86126,v1:172.21.15.83:6819/86126] conn(0x7fbde80725e0 0x7fbde80749f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 20 client.4278.objecter put_session s=0x7fbde8071f10 osd=5 1 2023-10-06T13:53:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 20 client.4278.objecter shutdown clearing up homeless session... 2023-10-06T13:53:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 10 client.4278.objecter successfully canceled tick 2023-10-06T13:53:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 1 -- 172.21.15.19:0/643282779 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbde806c210 msgr2=0x7fbde806e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 1 --2- 172.21.15.19:0/643282779 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbde806c210 0x7fbde806e6c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009990 tx=0x7fbe00009110 comp rx=0 tx=0).stop 2023-10-06T13:53:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 10 monclient: shutdown 2023-10-06T13:53:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 1 -- 172.21.15.19:0/643282779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe0413da90 msgr2=0x7fbe0413e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.565 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 1 --2- 172.21.15.19:0/643282779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe0413da90 0x7fbe0413e920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc036040 tx=0x7fbdfc004a40 comp rx=0 tx=0).stop 2023-10-06T13:53:19.565 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0acf0640 1 -- 172.21.15.19:0/643282779 reap_dead start 2023-10-06T13:53:19.565 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 1 -- 172.21.15.19:0/643282779 shutdown_connections 2023-10-06T13:53:19.565 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 1 -- 172.21.15.19:0/643282779 >> 172.21.15.19:0/643282779 conn(0x7fbe04065920 msgr2=0x7fbe040b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.565 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 1 -- 172.21.15.19:0/643282779 shutdown_connections 2023-10-06T13:53:19.565 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 1 -- 172.21.15.19:0/643282779 wait complete. 2023-10-06T13:53:19.565 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.537+0000 7fbe0c753640 1 librados: shutdown 2023-10-06T13:53:19.566 INFO:teuthology.orchestra.run.smithi019.stdout:55834574852 2023-10-06T13:53:19.566 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T13:53:19.567 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.543+0000 7faad67fc640 1 -- 172.21.15.19:0/170907513 <== osd.3 v2:172.21.15.19:6800/86196 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7faac40750d0 con 0x7faac4072610 2023-10-06T13:53:19.567 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.543+0000 7faad67fc640 10 client.4250.objecter ms_dispatch 0x7faae0000b90 command_reply(tid 1: 0 ) v1 2023-10-06T13:53:19.567 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.543+0000 7faad67fc640 10 client.4250.objecter _finish_command 1 = system:0 2023-10-06T13:53:19.567 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.543+0000 7faad67fc640 20 client.4250.objecter put_session s=0x7faac4071f40 osd=3 4 2023-10-06T13:53:19.567 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.543+0000 7faad67fc640 15 client.4250.objecter _session_command_op_remove 3 1 2023-10-06T13:53:19.568 INFO:teuthology.orchestra.run.smithi019.stdout:60129542150 2023-10-06T13:53:19.568 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T13:53:19.569 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.550+0000 7fa058ff9640 1 -- 172.21.15.19:0/3510967138 <== osd.0 v2:172.21.15.19:6816/86195 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa0480750a0 con 0x7fa0480725e0 2023-10-06T13:53:19.569 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.550+0000 7fa058ff9640 10 client.4284.objecter ms_dispatch 0x7fa05c000b90 command_reply(tid 1: 0 ) v1 2023-10-06T13:53:19.569 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.550+0000 7fa058ff9640 10 client.4284.objecter _finish_command 1 = system:0 2023-10-06T13:53:19.569 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.550+0000 7fa058ff9640 20 client.4284.objecter put_session s=0x7fa048071f10 osd=0 4 2023-10-06T13:53:19.569 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.550+0000 7fa058ff9640 15 client.4284.objecter _session_command_op_remove 0 1 2023-10-06T13:53:19.569 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.558+0000 7faae7e66640 10 client.4250.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-10-06T13:53:19.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.558+0000 7faae7e66640 15 client.4250.objecter _session_command_op_assign -1 2 2023-10-06T13:53:19.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.558+0000 7faae7e66640 20 client.4250.objecter _get_session s=0x7faac4071f40 osd=3 3 2023-10-06T13:53:19.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.558+0000 7faae7e66640 20 client.4250.objecter put_session s=0x7faac4071f40 osd=3 3 2023-10-06T13:53:19.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.558+0000 7faae7e66640 20 client.4250.objecter _get_session s=0x7faac4071f40 osd=3 3 2023-10-06T13:53:19.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.558+0000 7faae7e66640 15 client.4250.objecter _session_command_op_remove -1 2 2023-10-06T13:53:19.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.558+0000 7faae7e66640 20 client.4250.objecter get_session s=0x7faac4071f40 osd=3 3 2023-10-06T13:53:19.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.558+0000 7faae7e66640 15 client.4250.objecter _session_command_op_assign 3 2 2023-10-06T13:53:19.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.558+0000 7faae7e66640 20 client.4250.objecter put_session s=0x7faac4071f40 osd=3 4 2023-10-06T13:53:19.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.558+0000 7faae7e66640 10 client.4250.objecter _send_command 2 2023-10-06T13:53:19.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.558+0000 7faae7e66640 1 -- 172.21.15.19:0/170907513 --> [v2:172.21.15.19:6800/86196,v1:172.21.15.19:6801/86196] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faae00af0a0 con 0x7faac4072610 2023-10-06T13:53:19.571 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.564+0000 7fa062884640 10 client.4284.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-10-06T13:53:19.571 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7fa062884640 15 client.4284.objecter _session_command_op_assign -1 2 2023-10-06T13:53:19.571 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7fa062884640 20 client.4284.objecter _get_session s=0x7fa048071f10 osd=0 3 2023-10-06T13:53:19.571 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7fa062884640 20 client.4284.objecter put_session s=0x7fa048071f10 osd=0 3 2023-10-06T13:53:19.571 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7fa062884640 20 client.4284.objecter _get_session s=0x7fa048071f10 osd=0 3 2023-10-06T13:53:19.571 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7fa062884640 15 client.4284.objecter _session_command_op_remove -1 2 2023-10-06T13:53:19.572 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7fa062884640 20 client.4284.objecter get_session s=0x7fa048071f10 osd=0 3 2023-10-06T13:53:19.572 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7fa062884640 15 client.4284.objecter _session_command_op_assign 0 2 2023-10-06T13:53:19.572 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7fa062884640 20 client.4284.objecter put_session s=0x7fa048071f10 osd=0 4 2023-10-06T13:53:19.572 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7fa062884640 10 client.4284.objecter _send_command 2 2023-10-06T13:53:19.572 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7fa062884640 1 -- 172.21.15.19:0/3510967138 --> [v2:172.21.15.19:6816/86195,v1:172.21.15.19:6818/86195] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa05c0b1420 con 0x7fa0480725e0 2023-10-06T13:53:19.572 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faad67fc640 1 -- 172.21.15.19:0/170907513 <== osd.3 v2:172.21.15.19:6800/86196 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faad8001030 con 0x7faac4072610 2023-10-06T13:53:19.572 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faad67fc640 10 client.4250.objecter ms_dispatch 0x7faae0000b90 command_reply(tid 2: 0 ) v1 2023-10-06T13:53:19.572 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faad67fc640 10 client.4250.objecter _finish_command 2 = system:0 2023-10-06T13:53:19.573 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faad67fc640 20 client.4250.objecter put_session s=0x7faac4071f40 osd=3 4 2023-10-06T13:53:19.573 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faad67fc640 15 client.4250.objecter _session_command_op_remove 3 2 2023-10-06T13:53:19.574 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 10 librados: watch_flush enter 2023-10-06T13:53:19.574 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 10 librados: watch_flush exit 2023-10-06T13:53:19.574 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 10 client.4250.objecter close_session for osd.3 2023-10-06T13:53:19.574 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 1 -- 172.21.15.19:0/170907513 >> [v2:172.21.15.19:6800/86196,v1:172.21.15.19:6801/86196] conn(0x7faac4072610 msgr2=0x7faac4074a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.574 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 1 --2- 172.21.15.19:0/170907513 >> [v2:172.21.15.19:6800/86196,v1:172.21.15.19:6801/86196] conn(0x7faac4072610 0x7faac4074a20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.574 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 20 client.4250.objecter put_session s=0x7faac4071f40 osd=3 1 2023-10-06T13:53:19.574 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 20 client.4250.objecter shutdown clearing up homeless session... 2023-10-06T13:53:19.574 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 10 client.4250.objecter successfully canceled tick 2023-10-06T13:53:19.574 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 1 -- 172.21.15.19:0/170907513 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7faac406c240 msgr2=0x7faac406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.574 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 1 --2- 172.21.15.19:0/170907513 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7faac406c240 0x7faac406e6f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faadc009710 tx=0x7faadc006c90 comp rx=0 tx=0).stop 2023-10-06T13:53:19.575 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 10 monclient: shutdown 2023-10-06T13:53:19.575 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.575 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 1 -- 172.21.15.19:0/170907513 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faae00b37e0 msgr2=0x7faae01dd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.575 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 1 --2- 172.21.15.19:0/170907513 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7faae00b37e0 0x7faae01dd730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faad000c410 tx=0x7faad000c8d0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.575 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae6403640 1 -- 172.21.15.19:0/170907513 reap_dead start 2023-10-06T13:53:19.575 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 1 -- 172.21.15.19:0/170907513 shutdown_connections 2023-10-06T13:53:19.575 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 1 -- 172.21.15.19:0/170907513 >> 172.21.15.19:0/170907513 conn(0x7faae00b6e00 msgr2=0x7faae006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.575 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 1 -- 172.21.15.19:0/170907513 shutdown_connections 2023-10-06T13:53:19.575 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 1 -- 172.21.15.19:0/170907513 wait complete. 2023-10-06T13:53:19.575 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.565+0000 7faae7e66640 1 librados: shutdown 2023-10-06T13:53:19.576 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa058ff9640 1 -- 172.21.15.19:0/3510967138 <== osd.0 v2:172.21.15.19:6816/86195 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa054001f20 con 0x7fa0480725e0 2023-10-06T13:53:19.576 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa058ff9640 10 client.4284.objecter ms_dispatch 0x7fa05c000b90 command_reply(tid 2: 0 ) v1 2023-10-06T13:53:19.576 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa058ff9640 10 client.4284.objecter _finish_command 2 = system:0 2023-10-06T13:53:19.576 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa058ff9640 20 client.4284.objecter put_session s=0x7fa048071f10 osd=0 4 2023-10-06T13:53:19.576 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa058ff9640 15 client.4284.objecter _session_command_op_remove 0 2 2023-10-06T13:53:19.577 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa062884640 10 librados: watch_flush enter 2023-10-06T13:53:19.577 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa062884640 10 librados: watch_flush exit 2023-10-06T13:53:19.578 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa062884640 10 client.4284.objecter close_session for osd.0 2023-10-06T13:53:19.578 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa062884640 1 -- 172.21.15.19:0/3510967138 >> [v2:172.21.15.19:6816/86195,v1:172.21.15.19:6818/86195] conn(0x7fa0480725e0 msgr2=0x7fa0480749f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.578 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa062884640 1 --2- 172.21.15.19:0/3510967138 >> [v2:172.21.15.19:6816/86195,v1:172.21.15.19:6818/86195] conn(0x7fa0480725e0 0x7fa0480749f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.578 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa062884640 20 client.4284.objecter put_session s=0x7fa048071f10 osd=0 1 2023-10-06T13:53:19.578 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa062884640 20 client.4284.objecter shutdown clearing up homeless session... 2023-10-06T13:53:19.578 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa062884640 10 client.4284.objecter successfully canceled tick 2023-10-06T13:53:19.578 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa062884640 1 -- 172.21.15.19:0/3510967138 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa04806c210 msgr2=0x7fa04806e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.578 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa062884640 1 --2- 172.21.15.19:0/3510967138 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa04806c210 0x7fa04806e6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa04c000f60 tx=0x7fa04c009040 comp rx=0 tx=0).stop 2023-10-06T13:53:19.578 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa062884640 10 monclient: shutdown 2023-10-06T13:53:19.578 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa062884640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.578 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa062884640 1 -- 172.21.15.19:0/3510967138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c13e2b0 msgr2=0x7fa05c1d9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.578 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.571+0000 7fa062884640 1 --2- 172.21.15.19:0/3510967138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c13e2b0 0x7fa05c1d9830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa05c06e050 tx=0x7fa0500046c0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.579 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.572+0000 7fa060e21640 1 -- 172.21.15.19:0/3510967138 reap_dead start 2023-10-06T13:53:19.579 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.572+0000 7fa062884640 1 -- 172.21.15.19:0/3510967138 shutdown_connections 2023-10-06T13:53:19.579 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.572+0000 7fa062884640 1 -- 172.21.15.19:0/3510967138 >> 172.21.15.19:0/3510967138 conn(0x7fa05c0b4fa0 msgr2=0x7fa05c0b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.579 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.572+0000 7fa062884640 1 -- 172.21.15.19:0/3510967138 shutdown_connections 2023-10-06T13:53:19.579 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.572+0000 7fa062884640 1 -- 172.21.15.19:0/3510967138 wait complete. 2023-10-06T13:53:19.579 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.572+0000 7fa062884640 1 librados: shutdown 2023-10-06T13:53:19.582 INFO:teuthology.orchestra.run.smithi019.stdout:64424509447 2023-10-06T13:53:19.582 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T13:53:19.589 INFO:teuthology.orchestra.run.smithi019.stdout:68719476743 2023-10-06T13:53:19.589 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T13:53:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.623+0000 7ffa97fff640 1 -- 172.21.15.19:0/2166371806 <== osd.6 v2:172.21.15.83:6802/86124 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ffa800750a0 con 0x7ffa800725e0 2023-10-06T13:53:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.623+0000 7ffa97fff640 10 client.4281.objecter ms_dispatch 0x7ffaa0000b90 command_reply(tid 1: 0 ) v1 2023-10-06T13:53:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.623+0000 7ffa97fff640 10 client.4281.objecter _finish_command 1 = system:0 2023-10-06T13:53:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.623+0000 7ffa97fff640 20 client.4281.objecter put_session s=0x7ffa80071f10 osd=6 4 2023-10-06T13:53:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.623+0000 7ffa97fff640 15 client.4281.objecter _session_command_op_remove 6 1 2023-10-06T13:53:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.656+0000 7ffaa567a640 10 client.4281.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-10-06T13:53:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.656+0000 7ffaa567a640 15 client.4281.objecter _session_command_op_assign -1 2 2023-10-06T13:53:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.656+0000 7ffaa567a640 20 client.4281.objecter _get_session s=0x7ffa80071f10 osd=6 3 2023-10-06T13:53:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.656+0000 7ffaa567a640 20 client.4281.objecter put_session s=0x7ffa80071f10 osd=6 3 2023-10-06T13:53:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.656+0000 7ffaa567a640 20 client.4281.objecter _get_session s=0x7ffa80071f10 osd=6 3 2023-10-06T13:53:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.656+0000 7ffaa567a640 15 client.4281.objecter _session_command_op_remove -1 2 2023-10-06T13:53:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.656+0000 7ffaa567a640 20 client.4281.objecter get_session s=0x7ffa80071f10 osd=6 3 2023-10-06T13:53:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.656+0000 7ffaa567a640 15 client.4281.objecter _session_command_op_assign 6 2 2023-10-06T13:53:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.656+0000 7ffaa567a640 20 client.4281.objecter put_session s=0x7ffa80071f10 osd=6 4 2023-10-06T13:53:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.656+0000 7ffaa567a640 10 client.4281.objecter _send_command 2 2023-10-06T13:53:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.656+0000 7ffaa567a640 1 -- 172.21.15.19:0/2166371806 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffaa00aee30 con 0x7ffa800725e0 2023-10-06T13:53:19.664 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.662+0000 7ffa97fff640 1 -- 172.21.15.19:0/2166371806 <== osd.6 v2:172.21.15.83:6802/86124 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffa98009450 con 0x7ffa800725e0 2023-10-06T13:53:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.662+0000 7ffa97fff640 10 client.4281.objecter ms_dispatch 0x7ffaa0000b90 command_reply(tid 2: 0 ) v1 2023-10-06T13:53:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.662+0000 7ffa97fff640 10 client.4281.objecter _finish_command 2 = system:0 2023-10-06T13:53:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.662+0000 7ffa97fff640 20 client.4281.objecter put_session s=0x7ffa80071f10 osd=6 4 2023-10-06T13:53:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.662+0000 7ffa97fff640 15 client.4281.objecter _session_command_op_remove 6 2 2023-10-06T13:53:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.663+0000 7ffaa567a640 10 librados: watch_flush enter 2023-10-06T13:53:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.663+0000 7ffaa567a640 10 librados: watch_flush exit 2023-10-06T13:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.663+0000 7ffaa567a640 10 client.4281.objecter close_session for osd.6 2023-10-06T13:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.663+0000 7ffaa567a640 1 -- 172.21.15.19:0/2166371806 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x7ffa800725e0 msgr2=0x7ffa800749f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.663+0000 7ffaa567a640 1 --2- 172.21.15.19:0/2166371806 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x7ffa800725e0 0x7ffa800749f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.663+0000 7ffaa567a640 20 client.4281.objecter put_session s=0x7ffa80071f10 osd=6 1 2023-10-06T13:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.663+0000 7ffaa567a640 20 client.4281.objecter shutdown clearing up homeless session... 2023-10-06T13:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.663+0000 7ffaa567a640 10 client.4281.objecter successfully canceled tick 2023-10-06T13:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.664+0000 7ffaa567a640 1 -- 172.21.15.19:0/2166371806 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ffa8006c210 msgr2=0x7ffa8006e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.664+0000 7ffaa567a640 1 --2- 172.21.15.19:0/2166371806 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ffa8006c210 0x7ffa8006e6c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffaa01450a0 tx=0x7ffa88009040 comp rx=0 tx=0).stop 2023-10-06T13:53:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.664+0000 7ffaa567a640 10 monclient: shutdown 2023-10-06T13:53:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.664+0000 7ffaa567a640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.664+0000 7ffaa567a640 1 -- 172.21.15.19:0/2166371806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00b00c0 msgr2=0x7ffaa00aa0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.664+0000 7ffaa567a640 1 --2- 172.21.15.19:0/2166371806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00b00c0 0x7ffaa00aa0d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000c7f0 tx=0x7ffa9000ccb0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.664+0000 7ffa9f7fe640 1 -- 172.21.15.19:0/2166371806 reap_dead start 2023-10-06T13:53:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.664+0000 7ffaa567a640 1 -- 172.21.15.19:0/2166371806 shutdown_connections 2023-10-06T13:53:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.664+0000 7ffaa567a640 1 -- 172.21.15.19:0/2166371806 >> 172.21.15.19:0/2166371806 conn(0x7ffaa00b6e00 msgr2=0x7ffaa00b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.667+0000 7ffaa567a640 1 -- 172.21.15.19:0/2166371806 shutdown_connections 2023-10-06T13:53:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.667+0000 7ffaa567a640 1 -- 172.21.15.19:0/2166371806 wait complete. 2023-10-06T13:53:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.667+0000 7ffaa567a640 1 librados: shutdown 2023-10-06T13:53:19.688 INFO:teuthology.orchestra.run.smithi019.stdout:60129542149 2023-10-06T13:53:19.689 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T13:53:19.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.726+0000 7f1254b3c640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.726+0000 7f1254b3c640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.726+0000 7f1254b3c640 10 monclient: monmap: 2023-10-06T13:53:19.728 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.728 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.728 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.727130+0000 2023-10-06T13:53:19.729 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.727130+0000 2023-10-06T13:53:19.729 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.729 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.729 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.729 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.729 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.729 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.729 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.729 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.727+0000 7f1254b3c640 1 Processor -- start 2023-10-06T13:53:19.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.727+0000 7f1254b3c640 1 -- start start 2023-10-06T13:53:19.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.727+0000 7f1254b3c640 10 monclient: init 2023-10-06T13:53:19.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f23d4720640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f23d4720640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f23d4720640 10 monclient: monmap: 2023-10-06T13:53:19.731 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.731 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.731 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.729139+0000 2023-10-06T13:53:19.731 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.729139+0000 2023-10-06T13:53:19.731 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.732 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.732 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.732 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.732 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.732 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.732 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.732 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.732 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2023-10-06T13:53:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12500b3990 0x7f12500af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 10 monclient(hunting): picked mon.noname-a con 0x7f12500b3990 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f12500b3d60 0x7f1250141500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 10 monclient(hunting): picked mon.noname-b con 0x7f12500b3d60 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1250141a40 0x7f12500b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 10 monclient(hunting): picked mon.noname-c con 0x7f1250141a40 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1250140dd0 con 0x7f12500b3990 2023-10-06T13:53:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12500aebe0 con 0x7f1250141a40 2023-10-06T13:53:19.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1250141070 con 0x7f12500b3d60 2023-10-06T13:53:19.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.728+0000 7f1254b3c640 10 monclient(hunting): authenticate will time out at 1327.915111s 2023-10-06T13:53:19.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.729+0000 7f124dd9b640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f12500b3d60 0x7f1250141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.729+0000 7f124e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12500b3990 0x7f12500af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124dd9b640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f12500b3d60 0x7f1250141500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:41320/0 (socket says 172.21.15.19:41320) 2023-10-06T13:53:19.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124dd9b640 1 -- 172.21.15.19:0/2347787813 learned_addr learned my addr 172.21.15.19:0/2347787813 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124dd9b640 10 monclient(hunting): get_auth_request con 0x7f12500b3d60 auth_method 0 2023-10-06T13:53:19.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124dd9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124dd9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124ed9d640 1 --2- 172.21.15.19:0/2347787813 >> v2:172.21.15.83:3300/0 conn(0x7f1250141a40 0x7f12500b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124ed9d640 10 monclient(hunting): get_auth_request con 0x7f1250141a40 auth_method 0 2023-10-06T13:53:19.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124ed9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124ed9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124e59c640 10 monclient(hunting): get_auth_request con 0x7f12500b3990 auth_method 0 2023-10-06T13:53:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124e59c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124e59c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.730+0000 7f124dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f124e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f124e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f124e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f124ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f124ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f124ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.733+0000 7f124dd9b640 10 monclient(hunting): handle_auth_done global_id 4253 payload 274 2023-10-06T13:53:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.733+0000 7f124dd9b640 1 -- 172.21.15.19:0/2347787813 >> v2:172.21.15.83:3300/0 conn(0x7f1250141a40 msgr2=0x7f12500b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.733+0000 7f124dd9b640 1 --2- 172.21.15.19:0/2347787813 >> v2:172.21.15.83:3300/0 conn(0x7f1250141a40 0x7f12500b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.731+0000 7f23d4720640 1 Processor -- start 2023-10-06T13:53:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.731+0000 7f23d4720640 1 -- start start 2023-10-06T13:53:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.731+0000 7f23d4720640 10 monclient: init 2023-10-06T13:53:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f23d4720640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f23d4720640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2023-10-06T13:53:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f23d4720640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f23cc0b3990 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f23d4720640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f23cc0b3d60 0x7f23cc0af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f23d4720640 10 monclient(hunting): picked mon.noname-c con 0x7f23cc0b3d60 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f23d4720640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f23cc0b00c0 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f23d4720640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23cc0aebe0 con 0x7f23cc0b3990 2023-10-06T13:53:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f23d4720640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23cc141070 con 0x7f23cc0b00c0 2023-10-06T13:53:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f23d4720640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f23d4720640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23cc140dd0 con 0x7f23cc0b3d60 2023-10-06T13:53:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f23d4720640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.732+0000 7f23d4720640 10 monclient(hunting): authenticate will time out at 1327.918733s 2023-10-06T13:53:19.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.733+0000 7f124dd9b640 1 -- 172.21.15.19:0/2347787813 >> v2:172.21.15.19:3300/0 conn(0x7f12500b3990 msgr2=0x7f12500af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7f124dd9b640 1 --2- 172.21.15.19:0/2347787813 >> v2:172.21.15.19:3300/0 conn(0x7f12500b3990 0x7f12500af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7f124dd9b640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7f124dd9b640 1 monclient: found mon.noname-b 2023-10-06T13:53:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7f23d2cbd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f23cc0b00c0 0x7f23cc14b600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:38332/0 (socket says 172.21.15.19:38332) 2023-10-06T13:53:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7f23d2cbd640 1 -- 172.21.15.19:0/3714471588 learned_addr learned my addr 172.21.15.19:0/3714471588 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7f23d1cbb640 1 --2- 172.21.15.19:0/3714471588 >> v2:172.21.15.83:3300/0 conn(0x7f23cc0b3d60 0x7f23cc0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7f124dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7f23d1cbb640 10 monclient(hunting): get_auth_request con 0x7f23cc0b3d60 auth_method 0 2023-10-06T13:53:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7f23d1cbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7f23d1cbb640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7f23d1cbb640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7f124dd9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7f124dd9b640 1 -- 172.21.15.19:0/2347787813 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12500aee30 con 0x7f12500b3d60 2023-10-06T13:53:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7fc1c27e4640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7fc1c27e4640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.735+0000 7fc1c27e4640 10 monclient: monmap: 2023-10-06T13:53:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.736701+0000 2023-10-06T13:53:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.736701+0000 2023-10-06T13:53:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.747 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.747 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.747 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.747 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.747 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.747 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.747 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.736+0000 7f23d1cbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.736+0000 7f23d1cbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.736+0000 7f23d1cbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.736+0000 7fc1c27e4640 1 Processor -- start 2023-10-06T13:53:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7f5be386a640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7f5be386a640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7f5be386a640 10 monclient: monmap: 2023-10-06T13:53:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.738135+0000 2023-10-06T13:53:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.738135+0000 2023-10-06T13:53:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.750 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.750 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.750 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.736+0000 7fc1c27e4640 1 -- start start 2023-10-06T13:53:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.736+0000 7fc1c27e4640 10 monclient: init 2023-10-06T13:53:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7fc1c27e4640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7fc1c27e4640 10 monclient: _add_conns ranks=[2,0,3,1,4] 2023-10-06T13:53:19.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7fc1c27e4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc1bc0b3990 0x7fc1bc0af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7fc1c27e4640 10 monclient(hunting): picked mon.noname-b con 0x7fc1bc0b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7fc1c27e4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc1bc0b3d60 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:19.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7fc1c27e4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1bc0b00c0 0x7fc1bc0b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7fc1c27e4640 10 monclient(hunting): picked mon.noname-a con 0x7fc1bc0b00c0 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:19.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7fc1c27e4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1bc0aebe0 con 0x7fc1bc0b3d60 2023-10-06T13:53:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7fc1c27e4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7fc1c27e4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1bc0afd50 con 0x7fc1bc0b00c0 2023-10-06T13:53:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7fc1c27e4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7fc1c27e4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc140dd0 con 0x7fc1bc0b3990 2023-10-06T13:53:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7fc1c27e4640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.737+0000 7fc1c27e4640 10 monclient(hunting): authenticate will time out at 1327.924344s 2023-10-06T13:53:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.738+0000 7f5be386a640 1 Processor -- start 2023-10-06T13:53:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.738+0000 7f5be386a640 1 -- start start 2023-10-06T13:53:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.738+0000 7f5be386a640 10 monclient: init 2023-10-06T13:53:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.739+0000 7f5be386a640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.739+0000 7f5be386a640 10 monclient: _add_conns ranks=[2,0,1,4,3] 2023-10-06T13:53:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.739+0000 7f5be386a640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bdc0b3410 0x7f5bdc0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.739+0000 7f5be386a640 10 monclient(hunting): picked mon.noname-b con 0x7f5bdc0b3410 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.739+0000 7f5be386a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5bdc0b37e0 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7f5be386a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5bdc145430 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7f5be386a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bdc0b3ce0 con 0x7f5bdc0b37e0 2023-10-06T13:53:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7f5be386a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bdc0aeae0 con 0x7f5bdc145430 2023-10-06T13:53:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7f5be386a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7f5be386a640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0ae900 con 0x7f5bdc0b3410 2023-10-06T13:53:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7f5be386a640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7f5be386a640 10 monclient(hunting): authenticate will time out at 1327.926655s 2023-10-06T13:53:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.739+0000 7f23d1cbb640 10 monclient(hunting): handle_auth_done global_id 4330 payload 274 2023-10-06T13:53:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.739+0000 7f23d1cbb640 1 -- 172.21.15.19:0/3714471588 >> v1:172.21.15.83:6789/0 conn(0x7f23cc0b00c0 legacy=0x7f23cc14b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.739+0000 7f23d1cbb640 1 -- 172.21.15.19:0/3714471588 >> v1:172.21.15.19:6789/0 conn(0x7f23cc0b3990 legacy=0x7f23cc141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.739+0000 7f23d1cbb640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.739+0000 7f23d1cbb640 1 monclient: found mon.noname-c 2023-10-06T13:53:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.739+0000 7f23d1cbb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.739+0000 7f23d1cbb640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.739+0000 7f23d1cbb640 1 -- 172.21.15.19:0/3714471588 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc141c10 con 0x7f23cc0b3d60 2023-10-06T13:53:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7fc1c0d81640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1bc0b00c0 0x7fc1bc0b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7fc1c0d81640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1bc0b00c0 0x7fc1bc0b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38842/0 (socket says 172.21.15.19:38842) 2023-10-06T13:53:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7fc1c0d81640 1 -- 172.21.15.19:0/4104010293 learned_addr learned my addr 172.21.15.19:0/4104010293 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7fc1c0d81640 10 monclient(hunting): get_auth_request con 0x7fc1bc0b00c0 auth_method 0 2023-10-06T13:53:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7fc1c0d81640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7fc1c0d81640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7fc1c0d81640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.740+0000 7f124dd9b640 1 --2- 172.21.15.19:0/2347787813 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f12500b3d60 0x7f1250141500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f12500b1080 tx=0x7f124402b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb6a5a2e78e7a08e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.741+0000 7fc1c0d81640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.741+0000 7fc1c0d81640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.741+0000 7fc1c0d81640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.741+0000 7f78def20640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.741+0000 7f78def20640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.742+0000 7f78def20640 10 monclient: monmap: 2023-10-06T13:53:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.743015+0000 2023-10-06T13:53:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.743015+0000 2023-10-06T13:53:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.759 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.759 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.742+0000 7f5be1e07640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5bdc145430 0x7f5bdc145890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:38336/0 (socket says 172.21.15.19:38336) 2023-10-06T13:53:19.759 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.742+0000 7f5be1e07640 1 -- 172.21.15.19:0/4209663584 learned_addr learned my addr 172.21.15.19:0/4209663584 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 1 Processor -- start 2023-10-06T13:53:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 1 -- start start 2023-10-06T13:53:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 10 monclient: init 2023-10-06T13:53:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 10 monclient: _add_conns ranks=[1,2,3,4,0] 2023-10-06T13:53:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f78d80b3990 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f78d80b3d60 0x7f78d80af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 10 monclient(hunting): picked mon.noname-b con 0x7f78d80b3d60 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f78d80b00c0 0x7f78d80b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 10 monclient(hunting): picked mon.noname-a con 0x7f78d80b00c0 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d80aebe0 con 0x7f78d80b3990 2023-10-06T13:53:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d80afd50 con 0x7f78d80b00c0 2023-10-06T13:53:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d8140dd0 con 0x7f78d80b3d60 2023-10-06T13:53:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.743+0000 7f78def20640 10 monclient(hunting): authenticate will time out at 1327.930353s 2023-10-06T13:53:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.745+0000 7f23d1cbb640 1 --2- 172.21.15.19:0/3714471588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23cc0b3d60 0x7f23cc0af810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009d90 tx=0x7f23bc02be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d84e1a7c7e2f4f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.746+0000 7fc1c0d81640 10 monclient(hunting): handle_auth_done global_id 4290 payload 274 2023-10-06T13:53:19.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.746+0000 7fc1c0d81640 1 -- 172.21.15.19:0/4104010293 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc1bc0b3990 msgr2=0x7fc1bc0af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.746+0000 7fc1c0d81640 1 --2- 172.21.15.19:0/4104010293 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc1bc0b3990 0x7fc1bc0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.746+0000 7fc1c0d81640 1 -- 172.21.15.19:0/4104010293 >> v1:172.21.15.19:6789/0 conn(0x7fc1bc0b3d60 legacy=0x7fc1bc141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.746+0000 7fc1c0d81640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.746+0000 7fc1c0d81640 1 monclient: found mon.noname-a 2023-10-06T13:53:19.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.746+0000 7fc1c0d81640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.746+0000 7fc1c0d81640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.764 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.746+0000 7fc1c0d81640 1 -- 172.21.15.19:0/4104010293 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc141070 con 0x7fc1bc0b00c0 2023-10-06T13:53:19.764 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.746+0000 7f78dccbc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f78d80b3990 0x7f78d8141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:38338/0 (socket says 172.21.15.19:38338) 2023-10-06T13:53:19.764 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.746+0000 7f78dccbc640 1 -- 172.21.15.19:0/3031972222 learned_addr learned my addr 172.21.15.19:0/3031972222 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:19.764 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f5be1606640 1 --2- 172.21.15.19:0/4209663584 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bdc0b3410 0x7f5bdc0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f124d59a640 1 -- 172.21.15.19:0/2347787813 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12440317f0 con 0x7f12500b3d60 2023-10-06T13:53:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f124d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f124d59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f124d59a640 10 monclient: dump: 2023-10-06T13:53:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.768 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f124d59a640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f124d59a640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.769 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f124d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.769 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f124d59a640 1 -- 172.21.15.19:0/2347787813 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1244031950 con 0x7f12500b3d60 2023-10-06T13:53:19.769 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f124d59a640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.769 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f124d59a640 1 -- 172.21.15.19:0/2347787813 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12440317f0 con 0x7f12500b3d60 2023-10-06T13:53:19.769 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f124d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.770 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f124d59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:19.770 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f124d59a640 10 monclient: dump: 2023-10-06T13:53:19.770 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.770 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.770 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.770 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.770 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.771 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f1254b3c640 5 monclient: authenticate success, global_id 4253 2023-10-06T13:53:19.772 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f1254b3c640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.772 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.747+0000 7f1254b3c640 4 set_mon_vals no callback set 2023-10-06T13:53:19.772 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f5be1606640 10 monclient(hunting): get_auth_request con 0x7f5bdc0b3410 auth_method 0 2023-10-06T13:53:19.772 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f5be1606640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f5be1606640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f5be1606640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f5bd3fff640 1 -- 172.21.15.19:0/4209663584 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729972706 0 0) 0x7f5bdc0aeae0 con 0x7f5bdc145430 2023-10-06T13:53:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f5bd3fff640 10 monclient(hunting): _negotiate 2023-10-06T13:53:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f5bd3fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f5bd3fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f5bd3fff640 10 monclient(hunting): my global_id is 4336 2023-10-06T13:53:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f5bd3fff640 1 -- 172.21.15.19:0/4209663584 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bc8003520 con 0x7f5bdc145430 2023-10-06T13:53:19.775 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f1254b3c640 10 monclient: shutdown 2023-10-06T13:53:19.775 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f1254b3c640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.775 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f1254b3c640 1 -- 172.21.15.19:0/2347787813 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f12500b3d60 msgr2=0x7f1250141500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.775 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f1254b3c640 1 --2- 172.21.15.19:0/2347787813 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f12500b3d60 0x7f1250141500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f12500b1080 tx=0x7f124402b890 comp rx=0 tx=0).stop 2023-10-06T13:53:19.775 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f1254b3c640 1 -- 172.21.15.19:0/2347787813 shutdown_connections 2023-10-06T13:53:19.776 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f1254b3c640 1 --2- 172.21.15.19:0/2347787813 >> v2:172.21.15.83:3300/0 conn(0x7f1250141a40 0x7f12500b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.776 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f5be1606640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.776 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f5be1606640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.776 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f5be1606640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f1254b3c640 1 --2- 172.21.15.19:0/2347787813 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f12500b3d60 0x7f1250141500 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f1254b3c640 1 --2- 172.21.15.19:0/2347787813 >> v2:172.21.15.19:3300/0 conn(0x7f12500b3990 0x7f12500af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.748+0000 7f1254b3c640 1 -- 172.21.15.19:0/2347787813 >> 172.21.15.19:0/2347787813 conn(0x7f12500b6e00 msgr2=0x7f12500b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.749+0000 7f1254b3c640 1 -- 172.21.15.19:0/2347787813 shutdown_connections 2023-10-06T13:53:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.749+0000 7f1254b3c640 1 -- 172.21.15.19:0/2347787813 wait complete. 2023-10-06T13:53:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.750+0000 7f1254b3c640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.750+0000 7f1254b3c640 10 monclient: monmap: 2023-10-06T13:53:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.779 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.779 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.779 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.779 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.779 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.779 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.780 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.780 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.780 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.750+0000 7f78dd4bd640 1 --2- 172.21.15.19:0/3031972222 >> v2:172.21.15.19:3300/0 conn(0x7f78d80b00c0 0x7f78d80b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.750+0000 7f78dd4bd640 10 monclient(hunting): get_auth_request con 0x7f78d80b00c0 auth_method 0 2023-10-06T13:53:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.750+0000 7f78dd4bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.750+0000 7f78dd4bd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.750+0000 7f78dd4bd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.782 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.750+0000 7f1254b3c640 1 librados: starting msgr at 2023-10-06T13:53:19.782 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.750+0000 7f1254b3c640 1 librados: starting objecter 2023-10-06T13:53:19.782 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.750+0000 7f1254b3c640 1 Processor -- start 2023-10-06T13:53:19.782 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 1 -- start start 2023-10-06T13:53:19.782 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 1 librados: setting wanted keys 2023-10-06T13:53:19.782 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 1 librados: calling monclient init 2023-10-06T13:53:19.782 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 10 monclient: init 2023-10-06T13:53:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f78dd4bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f78dd4bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f78dd4bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T13:53:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12500b3990 0x7f12501da940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 10 monclient(hunting): picked mon.noname-b con 0x7f12500b3990 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12500b3d60 0x7f125006f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 10 monclient(hunting): picked mon.noname-a con 0x7f12500b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f1250141a40 0x7f12501dd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 10 monclient(hunting): picked mon.noname-c con 0x7f1250141a40 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12501da420 con 0x7f12500b3d60 2023-10-06T13:53:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125006fcb0 con 0x7f12500b3990 2023-10-06T13:53:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125006fe70 con 0x7f1250141a40 2023-10-06T13:53:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f1254b3c640 10 monclient(hunting): authenticate will time out at 1327.938249s 2023-10-06T13:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f23d14ba640 1 -- 172.21.15.19:0/3714471588 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc02cbd0 con 0x7f23cc0b3d60 2023-10-06T13:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f78d77fe640 1 -- 172.21.15.19:0/3031972222 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606637018 0 0) 0x7f78d80aebe0 con 0x7f78d80b3990 2023-10-06T13:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f78d77fe640 10 monclient(hunting): _negotiate 2023-10-06T13:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f78d77fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f78d77fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f78d77fe640 10 monclient(hunting): my global_id is 4339 2023-10-06T13:53:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f78d77fe640 1 -- 172.21.15.19:0/3031972222 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78c8003520 con 0x7f78d80b3990 2023-10-06T13:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f23d14ba640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f23d14ba640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f23d14ba640 10 monclient: dump: 2023-10-06T13:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f23d14ba640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f23d14ba640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f23d14ba640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f23d4720640 5 monclient: authenticate success, global_id 4330 2023-10-06T13:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f23d4720640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f23d14ba640 1 -- 172.21.15.19:0/3714471588 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc02cd30 con 0x7f23cc0b3d60 2023-10-06T13:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f23d14ba640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f5be1606640 10 monclient(hunting): handle_auth_done global_id 4256 payload 274 2023-10-06T13:53:19.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f5be1606640 1 -- 172.21.15.19:0/4209663584 >> v1:172.21.15.83:6789/0 conn(0x7f5bdc145430 legacy=0x7f5bdc145890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f5be1606640 1 -- 172.21.15.19:0/4209663584 >> v1:172.21.15.19:6789/0 conn(0x7f5bdc0b37e0 legacy=0x7f5bdc0afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f5be1606640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f5be1606640 1 monclient: found mon.noname-b 2023-10-06T13:53:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f5be1606640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f5be1606640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.751+0000 7f5be1606640 1 -- 172.21.15.19:0/4209663584 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc0aedc0 con 0x7f5bdc0b3410 2023-10-06T13:53:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f23d4720640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f23d4720640 4 set_mon_vals no callback set 2023-10-06T13:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.752+0000 7f23d14ba640 1 -- 172.21.15.19:0/3714471588 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc02cea0 con 0x7f23cc0b3d60 2023-10-06T13:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.753+0000 7f23d14ba640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.753+0000 7f23d14ba640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.753+0000 7f23d14ba640 10 monclient: dump: 2023-10-06T13:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.753+0000 7f23d4720640 10 monclient: shutdown 2023-10-06T13:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.753+0000 7f23d4720640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.753+0000 7f23d4720640 1 -- 172.21.15.19:0/3714471588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23cc0b3d60 msgr2=0x7f23cc0af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.753+0000 7f23d4720640 1 --2- 172.21.15.19:0/3714471588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23cc0b3d60 0x7f23cc0af810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009d90 tx=0x7f23bc02be20 comp rx=0 tx=0).stop 2023-10-06T13:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.753+0000 7f23d4720640 1 -- 172.21.15.19:0/3714471588 shutdown_connections 2023-10-06T13:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.753+0000 7f23d4720640 1 --2- 172.21.15.19:0/3714471588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23cc0b3d60 0x7f23cc0af810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.753+0000 7f23d4720640 1 -- 172.21.15.19:0/3714471588 >> 172.21.15.19:0/3714471588 conn(0x7f23cc0b6e00 msgr2=0x7f23cc0b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.753+0000 7f23d4720640 1 -- 172.21.15.19:0/3714471588 shutdown_connections 2023-10-06T13:53:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.753+0000 7f78d7fff640 1 --2- 172.21.15.19:0/3031972222 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f78d80b3d60 0x7f78d80af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.754+0000 7f78dd4bd640 10 monclient(hunting): handle_auth_done global_id 4293 payload 274 2023-10-06T13:53:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.754+0000 7f78dd4bd640 1 -- 172.21.15.19:0/3031972222 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f78d80b3d60 msgr2=0x7f78d80af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.754+0000 7f78dd4bd640 1 --2- 172.21.15.19:0/3031972222 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f78d80b3d60 0x7f78d80af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.754+0000 7f78dd4bd640 1 -- 172.21.15.19:0/3031972222 >> v1:172.21.15.83:6789/0 conn(0x7f78d80b3990 legacy=0x7f78d8141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.754+0000 7f78dd4bd640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.754+0000 7f78dd4bd640 1 monclient: found mon.noname-a 2023-10-06T13:53:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.754+0000 7f78dd4bd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.754+0000 7f78dd4bd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.754+0000 7f78dd4bd640 1 -- 172.21.15.19:0/3031972222 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d8141070 con 0x7f78d80b00c0 2023-10-06T13:53:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.753+0000 7f23d4720640 1 -- 172.21.15.19:0/3714471588 wait complete. 2023-10-06T13:53:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.754+0000 7f23d4720640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.754+0000 7f23d4720640 10 monclient: monmap: 2023-10-06T13:53:19.796 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.796 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.796 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.796 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.796 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.796 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.796 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.796 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.796 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.796 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.755+0000 7f23d4720640 1 librados: starting msgr at 2023-10-06T13:53:19.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.755+0000 7f23d4720640 1 librados: starting objecter 2023-10-06T13:53:19.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.755+0000 7f23d4720640 1 Processor -- start 2023-10-06T13:53:19.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.755+0000 7f23d4720640 1 -- start start 2023-10-06T13:53:19.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.755+0000 7f23d4720640 1 librados: setting wanted keys 2023-10-06T13:53:19.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.755+0000 7f23d4720640 1 librados: calling monclient init 2023-10-06T13:53:19.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.755+0000 7f23d4720640 10 monclient: init 2023-10-06T13:53:19.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 10 monclient: _add_conns ranks=[2,1,0] 2023-10-06T13:53:19.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23cc0b00c0 0x7f23cc1dd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 10 monclient(hunting): picked mon.noname-c con 0x7f23cc0b00c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23cc0b3990 0x7f23cc0abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 10 monclient(hunting): picked mon.noname-b con 0x7f23cc0b3990 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23cc0b3d60 0x7f23cc1d9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 10 monclient(hunting): picked mon.noname-a con 0x7f23cc0b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0acef0 con 0x7f23cc0b3d60 2023-10-06T13:53:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc1dda00 con 0x7f23cc0b3990 2023-10-06T13:53:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0ac3b0 con 0x7f23cc0b00c0 2023-10-06T13:53:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f23d4720640 10 monclient(hunting): authenticate will time out at 1327.942751s 2023-10-06T13:53:19.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.756+0000 7f5be1606640 1 --2- 172.21.15.19:0/4209663584 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bdc0b3410 0x7f5bdc0af910 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80097d0 tx=0x7f5bd802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d96ef928a41abd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.758+0000 7f124e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12500b3990 0x7f12501da940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.758+0000 7f124dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12500b3d60 0x7f125006f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.759+0000 7f78dd4bd640 1 --2- 172.21.15.19:0/3031972222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d80b00c0 0x7f78d80b0540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009d90 tx=0x7f78d002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c253a450fc73b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.760+0000 7f124e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12500b3990 0x7f12501da940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39410/0 (socket says 172.21.15.19:39410) 2023-10-06T13:53:19.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.760+0000 7f124e59c640 1 -- 172.21.15.19:0/711642149 learned_addr learned my addr 172.21.15.19:0/711642149 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.760+0000 7f124e59c640 10 monclient(hunting): get_auth_request con 0x7f12500b3990 auth_method 0 2023-10-06T13:53:19.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.760+0000 7f124e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.760+0000 7f124e59c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.760+0000 7f124e59c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.760+0000 7f124dd9b640 10 monclient(hunting): get_auth_request con 0x7f12500b3d60 auth_method 0 2023-10-06T13:53:19.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.760+0000 7f124dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.760+0000 7f124dd9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.760+0000 7f124dd9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.761+0000 7f124ed9d640 1 --2- 172.21.15.19:0/711642149 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f1250141a40 0x7f12501dd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.761+0000 7f23d1cbb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23cc0b3990 0x7f23cc0abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.761+0000 7f124e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.761+0000 7f124e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.761+0000 7f124e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.761+0000 7f124ed9d640 10 monclient(hunting): get_auth_request con 0x7f1250141a40 auth_method 0 2023-10-06T13:53:19.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.761+0000 7f23d1cbb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23cc0b3990 0x7f23cc0abe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39422/0 (socket says 172.21.15.19:39422) 2023-10-06T13:53:19.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.761+0000 7f23d1cbb640 1 -- 172.21.15.19:0/3700926455 learned_addr learned my addr 172.21.15.19:0/3700926455 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.761+0000 7f23d1cbb640 10 monclient(hunting): get_auth_request con 0x7f23cc0b3990 auth_method 0 2023-10-06T13:53:19.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.761+0000 7f23d1cbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.761+0000 7f23d1cbb640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.761+0000 7f23d1cbb640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.761+0000 7f124ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.762+0000 7f124ed9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.762+0000 7f124ed9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.762+0000 7f23d2cbd640 1 --2- 172.21.15.19:0/3700926455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23cc0b3d60 0x7f23cc1d9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.762+0000 7f124dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.762+0000 7f124dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.762+0000 7f124dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.763+0000 7f23d24bc640 1 --2- 172.21.15.19:0/3700926455 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23cc0b00c0 0x7f23cc1dd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.763+0000 7f124ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.763+0000 7f124ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.763+0000 7f124ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.764+0000 7f124e59c640 10 monclient(hunting): handle_auth_done global_id 4342 payload 824 2023-10-06T13:53:19.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.764+0000 7f124e59c640 1 -- 172.21.15.19:0/711642149 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f1250141a40 msgr2=0x7f12501dd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.764+0000 7f124e59c640 1 --2- 172.21.15.19:0/711642149 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f1250141a40 0x7f12501dd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.764+0000 7f124e59c640 1 -- 172.21.15.19:0/711642149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12500b3d60 msgr2=0x7f125006f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.764+0000 7f124e59c640 1 --2- 172.21.15.19:0/711642149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12500b3d60 0x7f125006f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.764+0000 7f124e59c640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.764+0000 7f124e59c640 1 monclient: found mon.noname-b 2023-10-06T13:53:19.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.764+0000 7f124e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.764+0000 7f124e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.764+0000 7f124e59c640 1 -- 172.21.15.19:0/711642149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12440094d0 con 0x7f12500b3990 2023-10-06T13:53:19.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.764+0000 7f23d1cbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.764+0000 7f23d1cbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.812 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.764+0000 7f23d1cbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.812 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5bd3fff640 1 -- 172.21.15.19:0/4209663584 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd802d040 con 0x7f5bdc0b3410 2023-10-06T13:53:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5bd3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5bd3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5bd3fff640 10 monclient: dump: 2023-10-06T13:53:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.815 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.815 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5bd3fff640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.815 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5bd3fff640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.815 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5bd3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.815 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5bd3fff640 1 -- 172.21.15.19:0/4209663584 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd8031810 con 0x7f5bdc0b3410 2023-10-06T13:53:19.815 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5bd3fff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.815 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5bd3fff640 1 -- 172.21.15.19:0/4209663584 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8031ba0 con 0x7f5bdc0b3410 2023-10-06T13:53:19.815 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5bd3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.815 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5bd3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5bd3fff640 10 monclient: dump: 2023-10-06T13:53:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.817 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.817 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.817 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.817 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5be386a640 5 monclient: authenticate success, global_id 4256 2023-10-06T13:53:19.817 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5be386a640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.817 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f5be386a640 4 set_mon_vals no callback set 2023-10-06T13:53:19.817 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f23d24bc640 10 monclient(hunting): get_auth_request con 0x7f23cc0b00c0 auth_method 0 2023-10-06T13:53:19.817 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f23d24bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.818 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f23d24bc640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.818 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.765+0000 7f23d24bc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.766+0000 7f5be386a640 10 monclient: shutdown 2023-10-06T13:53:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.766+0000 7f5be386a640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.766+0000 7f5be386a640 1 -- 172.21.15.19:0/4209663584 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bdc0b3410 msgr2=0x7f5bdc0af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.766+0000 7f5be386a640 1 --2- 172.21.15.19:0/4209663584 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bdc0b3410 0x7f5bdc0af910 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80097d0 tx=0x7f5bd802b8b0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.766+0000 7f5be386a640 1 -- 172.21.15.19:0/4209663584 shutdown_connections 2023-10-06T13:53:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.766+0000 7f5be386a640 1 --2- 172.21.15.19:0/4209663584 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bdc0b3410 0x7f5bdc0af910 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.766+0000 7f5be386a640 1 -- 172.21.15.19:0/4209663584 >> 172.21.15.19:0/4209663584 conn(0x7f5bdc0b6e00 msgr2=0x7f5bdc0b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.766+0000 7f5be386a640 1 -- 172.21.15.19:0/4209663584 shutdown_connections 2023-10-06T13:53:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.766+0000 7f5be386a640 1 -- 172.21.15.19:0/4209663584 wait complete. 2023-10-06T13:53:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.767+0000 7f5be386a640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.767+0000 7f5be386a640 10 monclient: monmap: 2023-10-06T13:53:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.821 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.768+0000 7f5be386a640 1 librados: starting msgr at 2023-10-06T13:53:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.768+0000 7f5be386a640 1 librados: starting objecter 2023-10-06T13:53:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.768+0000 7f5be386a640 1 Processor -- start 2023-10-06T13:53:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.768+0000 7f5be386a640 1 -- start start 2023-10-06T13:53:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.768+0000 7f5be386a640 1 librados: setting wanted keys 2023-10-06T13:53:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.768+0000 7f5be386a640 1 librados: calling monclient init 2023-10-06T13:53:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.768+0000 7f5be386a640 10 monclient: init 2023-10-06T13:53:19.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.769+0000 7f23d1cbb640 10 monclient(hunting): handle_auth_done global_id 4345 payload 824 2023-10-06T13:53:19.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.769+0000 7f23d1cbb640 1 -- 172.21.15.19:0/3700926455 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23cc0b00c0 msgr2=0x7f23cc1dd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.769+0000 7f23d1cbb640 1 --2- 172.21.15.19:0/3700926455 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23cc0b00c0 0x7f23cc1dd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.769+0000 7f23d1cbb640 1 -- 172.21.15.19:0/3700926455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23cc0b3d60 msgr2=0x7f23cc1d9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.769+0000 7f23d1cbb640 1 --2- 172.21.15.19:0/3700926455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23cc0b3d60 0x7f23cc1d9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.769+0000 7f23d1cbb640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.770+0000 7f5be386a640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.769+0000 7f23d1cbb640 1 monclient: found mon.noname-b 2023-10-06T13:53:19.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.770+0000 7f5be386a640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:53:19.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.770+0000 7f5be386a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bdc0b3410 0x7f5bdc0aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.769+0000 7f23d1cbb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.769+0000 7f23d1cbb640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.769+0000 7f23d1cbb640 1 -- 172.21.15.19:0/3700926455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc009a70 con 0x7f23cc0b3990 2023-10-06T13:53:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.770+0000 7f5be386a640 10 monclient(hunting): picked mon.noname-a con 0x7f5bdc0b3410 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.770+0000 7f5be386a640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bdc0b37e0 0x7f5bdc1274c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.770+0000 7f5be386a640 10 monclient(hunting): picked mon.noname-c con 0x7f5bdc0b37e0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.770+0000 7f5be386a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bdc145430 0x7f5bdc1dd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.770+0000 7f124e59c640 1 --2- 172.21.15.19:0/711642149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12500b3990 0x7f12501da940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f124000e890 tx=0x7f124000ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.771+0000 7f5be386a640 10 monclient(hunting): picked mon.noname-b con 0x7f5bdc145430 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.771+0000 7f5be386a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.771+0000 7f5be386a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc1dd200 con 0x7f5bdc0b3410 2023-10-06T13:53:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.771+0000 7f5be386a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.771+0000 7f5be386a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc1dd4e0 con 0x7f5bdc145430 2023-10-06T13:53:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.771+0000 7f5be386a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.771+0000 7f5be386a640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc127f40 con 0x7f5bdc0b37e0 2023-10-06T13:53:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.771+0000 7f5be386a640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.771+0000 7f5be386a640 10 monclient(hunting): authenticate will time out at 1327.957558s 2023-10-06T13:53:19.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.771+0000 7f123effd640 10 client.?.objecter ms_handle_connect 0x7f12500b3990 2023-10-06T13:53:19.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.771+0000 7f123effd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.772+0000 7f23d1cbb640 1 --2- 172.21.15.19:0/3700926455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23cc0b3990 0x7f23cc0abe70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009d90 tx=0x7f23bc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.772+0000 7f23caffd640 10 client.?.objecter ms_handle_connect 0x7f23cc0b3990 2023-10-06T13:53:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.772+0000 7f23caffd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.832 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.776+0000 7f123effd640 1 -- 172.21.15.19:0/711642149 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124000c940 con 0x7f12500b3990 2023-10-06T13:53:19.832 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.776+0000 7f123effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.832 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.776+0000 7f123effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:19.832 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.776+0000 7f123effd640 10 monclient: dump: 2023-10-06T13:53:19.832 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.833 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.776+0000 7f123effd640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.776+0000 7f123effd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.776+0000 7f123effd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.776+0000 7f1254b3c640 5 monclient: authenticate success, global_id 4342 2023-10-06T13:53:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.776+0000 7f1254b3c640 10 monclient: _renew_subs 2023-10-06T13:53:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.776+0000 7f1254b3c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.776+0000 7f1254b3c640 1 -- 172.21.15.19:0/711642149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12501d9e70 con 0x7f12500b3990 2023-10-06T13:53:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f1254b3c640 10 client.4342.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f1254b3c640 10 monclient: _renew_subs 2023-10-06T13:53:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f1254b3c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f1254b3c640 1 -- 172.21.15.19:0/711642149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12501dda90 con 0x7f12500b3990 2023-10-06T13:53:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f1254b3c640 1 librados: init done 2023-10-06T13:53:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f123effd640 1 -- 172.21.15.19:0/711642149 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124000caa0 con 0x7f12500b3990 2023-10-06T13:53:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78d77fe640 1 -- 172.21.15.19:0/3031972222 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d002cbf0 con 0x7f78d80b00c0 2023-10-06T13:53:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78d77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78d77fe640 10 monclient: dump: 2023-10-06T13:53:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.839 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78d77fe640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78d77fe640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78d77fe640 1 -- 172.21.15.19:0/3031972222 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d002cd50 con 0x7f78d80b00c0 2023-10-06T13:53:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78d77fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78d77fe640 1 -- 172.21.15.19:0/3031972222 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0034660 con 0x7f78d80b00c0 2023-10-06T13:53:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78def20640 5 monclient: authenticate success, global_id 4293 2023-10-06T13:53:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78def20640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78def20640 4 set_mon_vals no callback set 2023-10-06T13:53:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78d77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f78d77fe640 10 monclient: dump: 2023-10-06T13:53:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.842 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f123effd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f123effd640 1 -- 172.21.15.19:0/711642149 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124000ce30 con 0x7f12500b3990 2023-10-06T13:53:19.844 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f78def20640 10 monclient: shutdown 2023-10-06T13:53:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f78def20640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f78def20640 1 -- 172.21.15.19:0/3031972222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d80b00c0 msgr2=0x7f78d80b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f78def20640 1 --2- 172.21.15.19:0/3031972222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d80b00c0 0x7f78d80b0540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009d90 tx=0x7f78d002be40 comp rx=0 tx=0).stop 2023-10-06T13:53:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f78def20640 1 -- 172.21.15.19:0/3031972222 shutdown_connections 2023-10-06T13:53:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f78def20640 1 --2- 172.21.15.19:0/3031972222 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f78d80b3d60 0x7f78d80af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f78def20640 1 --2- 172.21.15.19:0/3031972222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d80b00c0 0x7f78d80b0540 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f78def20640 1 -- 172.21.15.19:0/3031972222 >> 172.21.15.19:0/3031972222 conn(0x7f78d80b6e00 msgr2=0x7f78d80b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f5be0e05640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bdc0b37e0 0x7f5bdc1274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f5be0e05640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bdc0b37e0 0x7f5bdc1274c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:41360/0 (socket says 172.21.15.19:41360) 2023-10-06T13:53:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f5be0e05640 1 -- 172.21.15.19:0/1355777657 learned_addr learned my addr 172.21.15.19:0/1355777657 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f5be0e05640 10 monclient(hunting): get_auth_request con 0x7f5bdc0b37e0 auth_method 0 2023-10-06T13:53:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f5be0e05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f5be0e05640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f5be0e05640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f123effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.848 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f123effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:19.848 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.777+0000 7f123effd640 10 monclient: dump: 2023-10-06T13:53:19.848 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.848 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.848 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.848 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.848 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.849 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.779+0000 7f5be1e07640 1 --2- 172.21.15.19:0/1355777657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bdc145430 0x7f5bdc1dd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.778+0000 7f124cd99640 4 set_mon_vals no callback set 2023-10-06T13:53:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.779+0000 7f5be0e05640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.779+0000 7f5be0e05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.779+0000 7f5be0e05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.779+0000 7f78def20640 1 -- 172.21.15.19:0/3031972222 shutdown_connections 2023-10-06T13:53:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.780+0000 7f1254b3c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.779+0000 7f78def20640 1 -- 172.21.15.19:0/3031972222 wait complete. 2023-10-06T13:53:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.780+0000 7f1254b3c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.780+0000 7f1254b3c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.780+0000 7f1254b3c640 1 -- 172.21.15.19:0/711642149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125006f110 con 0x7f12500b3990 2023-10-06T13:53:19.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.780+0000 7f78def20640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.780+0000 7f78def20640 10 monclient: monmap: 2023-10-06T13:53:19.853 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.853 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.853 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.853 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.853 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.853 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.853 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.853 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.853 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.854 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.781+0000 7f78def20640 1 librados: starting msgr at 2023-10-06T13:53:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.781+0000 7f78def20640 1 librados: starting objecter 2023-10-06T13:53:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.781+0000 7f78def20640 1 Processor -- start 2023-10-06T13:53:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.781+0000 7f78def20640 1 -- start start 2023-10-06T13:53:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.781+0000 7f78def20640 1 librados: setting wanted keys 2023-10-06T13:53:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.781+0000 7f78def20640 1 librados: calling monclient init 2023-10-06T13:53:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.781+0000 7f78def20640 10 monclient: init 2023-10-06T13:53:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.781+0000 7f5be1606640 1 --2- 172.21.15.19:0/1355777657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bdc0b3410 0x7f5bdc0aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.856 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.782+0000 7f78def20640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.856 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.782+0000 7f78def20640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T13:53:19.856 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.782+0000 7f78def20640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78d80b00c0 0x7f78d806f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.856 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.782+0000 7f78def20640 10 monclient(hunting): picked mon.noname-b con 0x7f78d80b00c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.782+0000 7f78def20640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d80b3990 0x7f78d80a8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.782+0000 7f78def20640 10 monclient(hunting): picked mon.noname-a con 0x7f78d80b3990 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.783+0000 7f78def20640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f78d80b3d60 0x7f78d81da6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.783+0000 7f78def20640 10 monclient(hunting): picked mon.noname-c con 0x7f78d80b3d60 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.783+0000 7f78def20640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.783+0000 7f78def20640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d81da1f0 con 0x7f78d80b3990 2023-10-06T13:53:19.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.783+0000 7f78def20640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.783+0000 7f78def20640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d806fd10 con 0x7f78d80b00c0 2023-10-06T13:53:19.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.783+0000 7f78def20640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.783+0000 7f78def20640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d806e710 con 0x7f78d80b3d60 2023-10-06T13:53:19.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.783+0000 7f78def20640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.783+0000 7f78def20640 10 monclient(hunting): authenticate will time out at 1327.970177s 2023-10-06T13:53:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.782+0000 7f5be1606640 10 monclient(hunting): get_auth_request con 0x7f5bdc0b3410 auth_method 0 2023-10-06T13:53:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.782+0000 7f5be1606640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.782+0000 7f5be1606640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.782+0000 7f5be1606640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.784+0000 7f5be1606640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.784+0000 7f5be1606640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.784+0000 7f5be1606640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.784+0000 7f5be0e05640 10 monclient(hunting): handle_auth_done global_id 4265 payload 824 2023-10-06T13:53:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.784+0000 7f5be0e05640 1 -- 172.21.15.19:0/1355777657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bdc145430 msgr2=0x7f5bdc1dd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.784+0000 7f5be0e05640 1 --2- 172.21.15.19:0/1355777657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bdc145430 0x7f5bdc1dd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.784+0000 7f5be0e05640 1 -- 172.21.15.19:0/1355777657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bdc0b3410 msgr2=0x7f5bdc0aa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.784+0000 7f5be0e05640 1 --2- 172.21.15.19:0/1355777657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bdc0b3410 0x7f5bdc0aa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.784+0000 7f5be0e05640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.784+0000 7f5be0e05640 1 monclient: found mon.noname-c 2023-10-06T13:53:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.784+0000 7f5be0e05640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.784+0000 7f5be0e05640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.784+0000 7f5be0e05640 1 -- 172.21.15.19:0/1355777657 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd8009400 con 0x7f5bdc0b37e0 2023-10-06T13:53:19.860 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.785+0000 7f23caffd640 1 -- 172.21.15.19:0/3700926455 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc02cea0 con 0x7f23cc0b3990 2023-10-06T13:53:19.860 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.785+0000 7f23caffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.860 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.785+0000 7f23caffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:19.860 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.785+0000 7f23caffd640 10 monclient: dump: 2023-10-06T13:53:19.860 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.860 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.861 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.785+0000 7f23caffd640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.785+0000 7f23caffd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.785+0000 7f8ba588f640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.785+0000 7f8ba588f640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.785+0000 7f23caffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.785+0000 7f8ba588f640 10 monclient: monmap: 2023-10-06T13:53:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.786937+0000 2023-10-06T13:53:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.786937+0000 2023-10-06T13:53:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.863 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.785+0000 7f23caffd640 1 -- 172.21.15.19:0/3700926455 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc03f420 con 0x7f23cc0b3990 2023-10-06T13:53:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.785+0000 7f23caffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.786+0000 7f23d0cb9640 4 set_mon_vals no callback set 2023-10-06T13:53:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.786+0000 7f23d4720640 5 monclient: authenticate success, global_id 4345 2023-10-06T13:53:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.786+0000 7f23d4720640 10 monclient: _renew_subs 2023-10-06T13:53:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.786+0000 7f23d4720640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.786+0000 7f23d4720640 1 -- 172.21.15.19:0/3700926455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc06fd70 con 0x7f23cc0b3990 2023-10-06T13:53:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.786+0000 7f23d4720640 10 client.4345.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.786+0000 7f23d4720640 10 monclient: _renew_subs 2023-10-06T13:53:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.786+0000 7f23d4720640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.786+0000 7f23d4720640 1 -- 172.21.15.19:0/3700926455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc1de2f0 con 0x7f23cc0b3990 2023-10-06T13:53:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.786+0000 7f23d4720640 1 librados: init done 2023-10-06T13:53:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.786+0000 7f23caffd640 1 -- 172.21.15.19:0/3700926455 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc03f790 con 0x7f23cc0b3990 2023-10-06T13:53:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.786+0000 7f23caffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.786+0000 7f23caffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.786+0000 7f23caffd640 10 monclient: dump: 2023-10-06T13:53:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.866 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.787+0000 7f8ba588f640 1 Processor -- start 2023-10-06T13:53:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.787+0000 7f8ba588f640 1 -- start start 2023-10-06T13:53:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.787+0000 7f8ba588f640 10 monclient: init 2023-10-06T13:53:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f78dd4bd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f78d80b3d60 0x7f78d81da6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f23d4720640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f23d4720640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f23d4720640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:19.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f23d4720640 1 -- 172.21.15.19:0/3700926455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc0678e0 con 0x7f23cc0b3990 2023-10-06T13:53:19.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f78dd4bd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f78d80b3d60 0x7f78d81da6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:41374/0 (socket says 172.21.15.19:41374) 2023-10-06T13:53:19.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f78dd4bd640 1 -- 172.21.15.19:0/765837724 learned_addr learned my addr 172.21.15.19:0/765837724 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f78dd4bd640 10 monclient(hunting): get_auth_request con 0x7f78d80b3d60 auth_method 0 2023-10-06T13:53:19.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f78dd4bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f8ba588f640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f8ba588f640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2023-10-06T13:53:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f8ba588f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8ba00aa8d0 0x7f8ba0070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f78dd4bd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f78dd4bd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f8ba588f640 10 monclient(hunting): picked mon.noname-c con 0x7f8ba00aa8d0 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f8ba588f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8ba00ae860 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f8ba588f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ba00ad810 0x7f8ba00aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f8ba588f640 10 monclient(hunting): picked mon.noname-a con 0x7f8ba00ad810 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f8ba588f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba00aaca0 con 0x7f8ba00ae860 2023-10-06T13:53:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f8ba588f640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f8ba588f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba00adc00 con 0x7f8ba00ad810 2023-10-06T13:53:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f8ba588f640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f8ba588f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba00af220 con 0x7f8ba00aa8d0 2023-10-06T13:53:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f8ba588f640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f8ba588f640 10 monclient(hunting): authenticate will time out at 1327.974828s 2023-10-06T13:53:19.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f78dd4bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f78dd4bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.788+0000 7f78dd4bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.789+0000 7f5be0e05640 1 --2- 172.21.15.19:0/1355777657 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bdc0b37e0 0x7f5bdc1274c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c440 tx=0x7f5bcc00c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.789+0000 7f5bd1ffb640 10 client.?.objecter ms_handle_connect 0x7f5bdc0b37e0 2023-10-06T13:53:19.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.789+0000 7f5bd1ffb640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.791+0000 7fc1c0d81640 1 --2- 172.21.15.19:0/4104010293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1bc0b00c0 0x7fc1bc0b0540 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009e50 tx=0x7fc1b402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d87de86f027c4be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.792+0000 7f78dd4bd640 10 monclient(hunting): handle_auth_done global_id 4268 payload 824 2023-10-06T13:53:19.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.792+0000 7f78dd4bd640 1 -- 172.21.15.19:0/765837724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78d80b00c0 msgr2=0x7f78d806f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.792+0000 7f78dd4bd640 1 --2- 172.21.15.19:0/765837724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78d80b00c0 0x7f78d806f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.792+0000 7f78dd4bd640 1 -- 172.21.15.19:0/765837724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d80b3990 msgr2=0x7f78d80a8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.792+0000 7f78dd4bd640 1 --2- 172.21.15.19:0/765837724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d80b3990 0x7f78d80a8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.792+0000 7f78dd4bd640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.792+0000 7f78dd4bd640 1 monclient: found mon.noname-c 2023-10-06T13:53:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.792+0000 7f78dd4bd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.792+0000 7f78dd4bd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.792+0000 7f78dd4bd640 1 -- 172.21.15.19:0/765837724 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d0009a70 con 0x7f78d80b3d60 2023-10-06T13:53:19.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.796+0000 7f8b9e7fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8ba00ae860 0x7f8ba00ad100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57940/0 (socket says 172.21.15.19:57940) 2023-10-06T13:53:19.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.796+0000 7f8b9e7fc640 1 -- 172.21.15.19:0/990190225 learned_addr learned my addr 172.21.15.19:0/990190225 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:19.875 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7f78dd4bd640 1 --2- 172.21.15.19:0/765837724 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f78d80b3d60 0x7f78d81da6d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009d90 tx=0x7f78d0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.875 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7f78d57fa640 10 client.?.objecter ms_handle_connect 0x7f78d80b3d60 2023-10-06T13:53:19.875 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7f78d57fa640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.875 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7f0cdbea4640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7f0cdbea4640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7f0cdbea4640 10 monclient: monmap: 2023-10-06T13:53:19.876 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.876 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.876 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.799584+0000 2023-10-06T13:53:19.877 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.799584+0000 2023-10-06T13:53:19.877 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.877 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.877 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.877 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.877 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.878 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.879 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1baffd640 1 -- 172.21.15.19:0/4104010293 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b402c9f0 con 0x7fc1bc0b00c0 2023-10-06T13:53:19.879 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1baffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1baffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1baffd640 10 monclient: dump: 2023-10-06T13:53:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.881 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1baffd640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1baffd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1baffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1c27e4640 5 monclient: authenticate success, global_id 4290 2023-10-06T13:53:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1c27e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1baffd640 1 -- 172.21.15.19:0/4104010293 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b402cb50 con 0x7fc1bc0b00c0 2023-10-06T13:53:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1baffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1c27e4640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1c27e4640 4 set_mon_vals no callback set 2023-10-06T13:53:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1baffd640 1 -- 172.21.15.19:0/4104010293 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40343f0 con 0x7fc1bc0b00c0 2023-10-06T13:53:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1baffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1baffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.798+0000 7fc1baffd640 10 monclient: dump: 2023-10-06T13:53:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.884 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f0cdbea4640 1 Processor -- start 2023-10-06T13:53:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f8b9f7fe640 1 --2- 172.21.15.19:0/990190225 >> v2:172.21.15.19:3300/0 conn(0x7f8ba00ad810 0x7f8ba00aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7fc1c27e4640 10 monclient: shutdown 2023-10-06T13:53:19.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7fc1c27e4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7fc1c27e4640 1 -- 172.21.15.19:0/4104010293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1bc0b00c0 msgr2=0x7fc1bc0b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7fc1c27e4640 1 --2- 172.21.15.19:0/4104010293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1bc0b00c0 0x7fc1bc0b0540 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009e50 tx=0x7fc1b402be50 comp rx=0 tx=0).stop 2023-10-06T13:53:19.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7fc1c27e4640 1 -- 172.21.15.19:0/4104010293 shutdown_connections 2023-10-06T13:53:19.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7fc1c27e4640 1 --2- 172.21.15.19:0/4104010293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1bc0b00c0 0x7fc1bc0b0540 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7fc1c27e4640 1 --2- 172.21.15.19:0/4104010293 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc1bc0b3990 0x7fc1bc0af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7fc1c27e4640 1 -- 172.21.15.19:0/4104010293 >> 172.21.15.19:0/4104010293 conn(0x7fc1bc0b6e00 msgr2=0x7fc1bc0b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f0cdbea4640 1 -- start start 2023-10-06T13:53:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7fc1c27e4640 1 -- 172.21.15.19:0/4104010293 shutdown_connections 2023-10-06T13:53:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f0cdbea4640 10 monclient: init 2023-10-06T13:53:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f5bd1ffb640 1 -- 172.21.15.19:0/1355777657 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc005480 con 0x7f5bdc0b37e0 2023-10-06T13:53:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f5bd1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f5bd1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f5bd1ffb640 10 monclient: dump: 2023-10-06T13:53:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.888 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f5bd1ffb640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f5bd1ffb640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f5bd1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f5bd1ffb640 1 -- 172.21.15.19:0/1355777657 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc0035f0 con 0x7f5bdc0b37e0 2023-10-06T13:53:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f5bd1ffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f5bd1ffb640 1 -- 172.21.15.19:0/1355777657 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0039c0 con 0x7f5bdc0b37e0 2023-10-06T13:53:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f5bd1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f5bd1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.799+0000 7f5bd1ffb640 10 monclient: dump: 2023-10-06T13:53:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.890 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.890 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.890 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.890 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.890 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7fc1c27e4640 1 -- 172.21.15.19:0/4104010293 wait complete. 2023-10-06T13:53:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f5be386a640 5 monclient: authenticate success, global_id 4265 2023-10-06T13:53:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f5be386a640 10 monclient: _renew_subs 2023-10-06T13:53:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f5be386a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f5be386a640 1 -- 172.21.15.19:0/1355777657 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bdc0ac400 con 0x7f5bdc0b37e0 2023-10-06T13:53:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f5bd37fe640 4 set_mon_vals no callback set 2023-10-06T13:53:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f5be386a640 10 client.4265.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f5be386a640 10 monclient: _renew_subs 2023-10-06T13:53:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f5be386a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f5be386a640 1 -- 172.21.15.19:0/1355777657 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc127a00 con 0x7f5bdc0b37e0 2023-10-06T13:53:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f5be386a640 1 librados: init done 2023-10-06T13:53:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7fc1c27e4640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7fc1c27e4640 10 monclient: monmap: 2023-10-06T13:53:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.893 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f0cdbea4640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f0cdbea4640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2023-10-06T13:53:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f0cdbea4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0cd40b3990 0x7f0cd40af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f0cdbea4640 10 monclient(hunting): picked mon.noname-b con 0x7f0cd40b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f0cdbea4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0cd40b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f0cdbea4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0cd40b00c0 0x7f0cd40b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f0cdbea4640 10 monclient(hunting): picked mon.noname-a con 0x7f0cd40b00c0 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.801+0000 7fc1c27e4640 1 librados: starting msgr at 2023-10-06T13:53:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.801+0000 7fc1c27e4640 1 librados: starting objecter 2023-10-06T13:53:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f0cdbea4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd40aebe0 con 0x7f0cd40b3d60 2023-10-06T13:53:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f0cdbea4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f0cdbea4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd40afd50 con 0x7f0cd40b00c0 2023-10-06T13:53:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.801+0000 7fc1c27e4640 1 Processor -- start 2023-10-06T13:53:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f0cdbea4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f0cdbea4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd4140dd0 con 0x7f0cd40b3990 2023-10-06T13:53:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f0cdbea4640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.800+0000 7f0cdbea4640 10 monclient(hunting): authenticate will time out at 1327.987224s 2023-10-06T13:53:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.801+0000 7fc1c27e4640 1 -- start start 2023-10-06T13:53:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.801+0000 7fc1c27e4640 1 librados: setting wanted keys 2023-10-06T13:53:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.801+0000 7fc1c27e4640 1 librados: calling monclient init 2023-10-06T13:53:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.801+0000 7fc1c27e4640 10 monclient: init 2023-10-06T13:53:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 10 monclient: _add_conns ranks=[2,1,0] 2023-10-06T13:53:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc1bc0b00c0 0x7fc1bc144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 10 monclient(hunting): picked mon.noname-c con 0x7fc1bc0b00c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1bc0b3990 0x7fc1bc13d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 10 monclient(hunting): picked mon.noname-b con 0x7fc1bc0b3990 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1bc0b3d60 0x7fc1bc137110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 10 monclient(hunting): picked mon.noname-a con 0x7fc1bc0b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0a8c40 con 0x7fc1bc0b3d60 2023-10-06T13:53:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0a8e90 con 0x7fc1bc0b3990 2023-10-06T13:53:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc13dff0 con 0x7fc1bc0b00c0 2023-10-06T13:53:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7fc1c27e4640 10 monclient(hunting): authenticate will time out at 1327.988634s 2023-10-06T13:53:19.900 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7f5be386a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.900 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7f5be386a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.900 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7f5be386a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:19.900 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7f5be386a640 1 -- 172.21.15.19:0/1355777657 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bdc0678e0 con 0x7f5bdc0b37e0 2023-10-06T13:53:19.901 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7f8b9f7fe640 10 monclient(hunting): get_auth_request con 0x7f8ba00ad810 auth_method 0 2023-10-06T13:53:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7f8b9f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7f8b9f7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7f8b9f7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.802+0000 7f8b9dffb640 1 -- 172.21.15.19:0/990190225 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225545022 0 0) 0x7f8ba00aaca0 con 0x7f8ba00ae860 2023-10-06T13:53:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.803+0000 7f8b9dffb640 10 monclient(hunting): _negotiate 2023-10-06T13:53:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.803+0000 7f8b9dffb640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.803+0000 7f8b9dffb640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.803+0000 7f8b9dffb640 10 monclient(hunting): my global_id is 4302 2023-10-06T13:53:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.803+0000 7f8b9dffb640 1 -- 172.21.15.19:0/990190225 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b88003520 con 0x7f8ba00ae860 2023-10-06T13:53:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.803+0000 7f8b9f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.803+0000 7f8b9f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.803+0000 7f8b9f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.805+0000 7f8b9f7fe640 10 monclient(hunting): handle_auth_done global_id 4305 payload 274 2023-10-06T13:53:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.805+0000 7f8b9f7fe640 1 -- 172.21.15.19:0/990190225 >> v2:172.21.15.83:3300/0 conn(0x7f8ba00aa8d0 msgr2=0x7f8ba0070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.805+0000 7f8b9f7fe640 1 --2- 172.21.15.19:0/990190225 >> v2:172.21.15.83:3300/0 conn(0x7f8ba00aa8d0 0x7f8ba0070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.805+0000 7f8b9f7fe640 1 -- 172.21.15.19:0/990190225 >> v1:172.21.15.19:6789/0 conn(0x7f8ba00ae860 legacy=0x7f8ba00ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.805+0000 7f8b9f7fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.805+0000 7f8b9f7fe640 1 monclient: found mon.noname-a 2023-10-06T13:53:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.805+0000 7f8b9f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.805+0000 7f8b9f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.805+0000 7f8b9f7fe640 1 -- 172.21.15.19:0/990190225 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba00ab9a0 con 0x7f8ba00ad810 2023-10-06T13:53:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.808+0000 7f8b9f7fe640 1 --2- 172.21.15.19:0/990190225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ba00ad810 0x7f8ba00aece0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00b4c10 tx=0x7f8b98030250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70864e71c0f8be9f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.814+0000 7f0cda441640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0cd40b00c0 0x7f0cd40b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.815+0000 7f0cda441640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0cd40b00c0 0x7f0cd40b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38916/0 (socket says 172.21.15.19:38916) 2023-10-06T13:53:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.815+0000 7f0cda441640 1 -- 172.21.15.19:0/1444540174 learned_addr learned my addr 172.21.15.19:0/1444540174 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.815+0000 7f0cda441640 10 monclient(hunting): get_auth_request con 0x7f0cd40b00c0 auth_method 0 2023-10-06T13:53:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.815+0000 7f0cda441640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.815+0000 7f0cda441640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.815+0000 7f0cda441640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.816+0000 7f0cda441640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.816+0000 7f0cda441640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.816+0000 7f0cda441640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8b9dffb640 1 -- 172.21.15.19:0/990190225 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b98038440 con 0x7f8ba00ad810 2023-10-06T13:53:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8b9dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8b9dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8b9dffb640 10 monclient: dump: 2023-10-06T13:53:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.908 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8b9dffb640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8b9dffb640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8b9dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8ba588f640 5 monclient: authenticate success, global_id 4305 2023-10-06T13:53:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8ba588f640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8b9dffb640 1 -- 172.21.15.19:0/990190225 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b980385a0 con 0x7f8ba00ad810 2023-10-06T13:53:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8b9dffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8ba588f640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8ba588f640 4 set_mon_vals no callback set 2023-10-06T13:53:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8b9dffb640 1 -- 172.21.15.19:0/990190225 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b98038440 con 0x7f8ba00ad810 2023-10-06T13:53:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8b9dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8b9dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8b9dffb640 10 monclient: dump: 2023-10-06T13:53:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.911 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7fc1c0d81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1bc0b3d60 0x7fc1bc137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8ba588f640 10 monclient: shutdown 2023-10-06T13:53:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8ba588f640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8ba588f640 1 -- 172.21.15.19:0/990190225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ba00ad810 msgr2=0x7f8ba00aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.817+0000 7f8ba588f640 1 --2- 172.21.15.19:0/990190225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ba00ad810 0x7f8ba00aece0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00b4c10 tx=0x7f8b98030250 comp rx=0 tx=0).stop 2023-10-06T13:53:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.818+0000 7f8ba588f640 1 -- 172.21.15.19:0/990190225 shutdown_connections 2023-10-06T13:53:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.818+0000 7f8ba588f640 1 --2- 172.21.15.19:0/990190225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ba00ad810 0x7f8ba00aece0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.818+0000 7f8ba588f640 1 --2- 172.21.15.19:0/990190225 >> v2:172.21.15.83:3300/0 conn(0x7f8ba00aa8d0 0x7f8ba0070ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.818+0000 7f8ba588f640 1 -- 172.21.15.19:0/990190225 >> 172.21.15.19:0/990190225 conn(0x7f8ba00b05d0 msgr2=0x7f8ba00b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.818+0000 7f8ba588f640 1 -- 172.21.15.19:0/990190225 shutdown_connections 2023-10-06T13:53:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.818+0000 7f8ba588f640 1 -- 172.21.15.19:0/990190225 wait complete. 2023-10-06T13:53:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.818+0000 7f8ba588f640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.818+0000 7f8ba588f640 10 monclient: monmap: 2023-10-06T13:53:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.916 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 1 librados: starting msgr at 2023-10-06T13:53:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 1 librados: starting objecter 2023-10-06T13:53:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 1 Processor -- start 2023-10-06T13:53:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 1 -- start start 2023-10-06T13:53:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 1 librados: setting wanted keys 2023-10-06T13:53:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 1 librados: calling monclient init 2023-10-06T13:53:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 10 monclient: init 2023-10-06T13:53:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T13:53:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ba00aa8d0 0x7f8ba00b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 10 monclient(hunting): picked mon.noname-b con 0x7f8ba00aa8d0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8ba00ad810 0x7f8ba00a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 10 monclient(hunting): picked mon.noname-c con 0x7f8ba00ad810 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ba00ae860 0x7f8ba00b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 10 monclient(hunting): picked mon.noname-a con 0x7f8ba00ae860 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba00b3c90 con 0x7f8ba00ae860 2023-10-06T13:53:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba00a8ee0 con 0x7f8ba00aa8d0 2023-10-06T13:53:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba013c550 con 0x7f8ba00ad810 2023-10-06T13:53:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.819+0000 7f8ba588f640 10 monclient(hunting): authenticate will time out at 1328.006278s 2023-10-06T13:53:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.820+0000 7f0cda441640 10 monclient(hunting): handle_auth_done global_id 4308 payload 274 2023-10-06T13:53:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.820+0000 7f0cda441640 1 -- 172.21.15.19:0/1444540174 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0cd40b3990 msgr2=0x7f0cd40af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.820+0000 7f0cda441640 1 --2- 172.21.15.19:0/1444540174 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0cd40b3990 0x7f0cd40af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.919 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.820+0000 7f0cda441640 1 -- 172.21.15.19:0/1444540174 >> v1:172.21.15.83:6789/0 conn(0x7f0cd40b3d60 legacy=0x7f0cd4141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:19.919 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.820+0000 7f0cda441640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.919 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.820+0000 7f0cda441640 1 monclient: found mon.noname-a 2023-10-06T13:53:19.919 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.820+0000 7f0cda441640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.919 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.820+0000 7f0cda441640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.919 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.820+0000 7f0cda441640 1 -- 172.21.15.19:0/1444540174 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd4141070 con 0x7f0cd40b00c0 2023-10-06T13:53:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.821+0000 7fc1c0d81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1bc0b3d60 0x7fc1bc137110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38932/0 (socket says 172.21.15.19:38932) 2023-10-06T13:53:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.821+0000 7fc1c0d81640 1 -- 172.21.15.19:0/2118078786 learned_addr learned my addr 172.21.15.19:0/2118078786 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.821+0000 7fc1c0d81640 10 monclient(hunting): get_auth_request con 0x7fc1bc0b3d60 auth_method 0 2023-10-06T13:53:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.821+0000 7fc1c0d81640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.821+0000 7fc1c0d81640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.821+0000 7fc1c0d81640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.822+0000 7fc1c0d81640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.822+0000 7fc1c0d81640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.822+0000 7fc1c0d81640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.825+0000 7fc1c0d81640 10 monclient(hunting): handle_auth_done global_id 4311 payload 824 2023-10-06T13:53:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.825+0000 7fc1c0d81640 1 -- 172.21.15.19:0/2118078786 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc1bc0b00c0 msgr2=0x7fc1bc144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.825+0000 7fc1c0d81640 1 --2- 172.21.15.19:0/2118078786 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc1bc0b00c0 0x7fc1bc144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.825+0000 7fc1c0d81640 1 -- 172.21.15.19:0/2118078786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1bc0b3990 msgr2=0x7fc1bc13d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.825+0000 7fc1c0d81640 1 --2- 172.21.15.19:0/2118078786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1bc0b3990 0x7fc1bc13d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.825+0000 7fc1c0d81640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.825+0000 7fc1c0d81640 1 monclient: found mon.noname-a 2023-10-06T13:53:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.825+0000 7fc1c0d81640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.825+0000 7fc1c0d81640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.825+0000 7fc1c0d81640 1 -- 172.21.15.19:0/2118078786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b4009b30 con 0x7fc1bc0b3d60 2023-10-06T13:53:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.827+0000 7f0cda441640 1 --2- 172.21.15.19:0/1444540174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cd40b00c0 0x7f0cd40b0540 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009870 tx=0x7f0ccc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db6427fdab71921 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.832+0000 7fc1c0d81640 1 --2- 172.21.15.19:0/2118078786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1bc0b3d60 0x7fc1bc137110 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4036040 tx=0x7fc1b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.832+0000 7fc1b8ff9640 10 client.?.objecter ms_handle_connect 0x7fc1bc0b3d60 2023-10-06T13:53:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.832+0000 7fc1b8ff9640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cd8c3e640 1 -- 172.21.15.19:0/1444540174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc0317c0 con 0x7f0cd40b00c0 2023-10-06T13:53:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cd8c3e640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cd8c3e640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cd8c3e640 10 monclient: dump: 2023-10-06T13:53:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.929 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.929 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.929 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.929 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.929 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.929 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.929 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.930 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.930 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cd8c3e640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cd8c3e640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cd8c3e640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cd8c3e640 1 -- 172.21.15.19:0/1444540174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc031920 con 0x7f0cd40b00c0 2023-10-06T13:53:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cd8c3e640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cd8c3e640 1 -- 172.21.15.19:0/1444540174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc031cb0 con 0x7f0cd40b00c0 2023-10-06T13:53:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cd8c3e640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cd8c3e640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cd8c3e640 10 monclient: dump: 2023-10-06T13:53:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.933 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cdbea4640 5 monclient: authenticate success, global_id 4308 2023-10-06T13:53:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cdbea4640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.833+0000 7f0cdbea4640 4 set_mon_vals no callback set 2023-10-06T13:53:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.834+0000 7f0cdbea4640 10 monclient: shutdown 2023-10-06T13:53:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.834+0000 7f0cdbea4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.834+0000 7f0cdbea4640 1 -- 172.21.15.19:0/1444540174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cd40b00c0 msgr2=0x7f0cd40b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.834+0000 7f0cdbea4640 1 --2- 172.21.15.19:0/1444540174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cd40b00c0 0x7f0cd40b0540 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009870 tx=0x7f0ccc02b860 comp rx=0 tx=0).stop 2023-10-06T13:53:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.834+0000 7f0cdbea4640 1 -- 172.21.15.19:0/1444540174 shutdown_connections 2023-10-06T13:53:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.834+0000 7f0cdbea4640 1 --2- 172.21.15.19:0/1444540174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cd40b00c0 0x7f0cd40b0540 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.834+0000 7f0cdbea4640 1 --2- 172.21.15.19:0/1444540174 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0cd40b3990 0x7f0cd40af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.834+0000 7f0cdbea4640 1 -- 172.21.15.19:0/1444540174 >> 172.21.15.19:0/1444540174 conn(0x7f0cd40b6e00 msgr2=0x7f0cd40b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.834+0000 7f0cdbea4640 1 -- 172.21.15.19:0/1444540174 shutdown_connections 2023-10-06T13:53:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.834+0000 7f0cdbea4640 1 -- 172.21.15.19:0/1444540174 wait complete. 2023-10-06T13:53:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.835+0000 7f0cdbea4640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.835+0000 7f0cdbea4640 10 monclient: monmap: 2023-10-06T13:53:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:19.935 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.835+0000 7f0cdbea4640 1 librados: starting msgr at 2023-10-06T13:53:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.835+0000 7f0cdbea4640 1 librados: starting objecter 2023-10-06T13:53:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.835+0000 7f0cdbea4640 1 Processor -- start 2023-10-06T13:53:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.835+0000 7f0cdbea4640 1 -- start start 2023-10-06T13:53:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.835+0000 7f0cdbea4640 1 librados: setting wanted keys 2023-10-06T13:53:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.835+0000 7f0cdbea4640 1 librados: calling monclient init 2023-10-06T13:53:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.835+0000 7f0cdbea4640 10 monclient: init 2023-10-06T13:53:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:53:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cd40b00c0 0x7f0cd4143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 10 monclient(hunting): picked mon.noname-a con 0x7f0cd40b00c0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0cd40b3990 0x7f0cd41ddb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 10 monclient(hunting): picked mon.noname-c con 0x7f0cd40b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cd40b3d60 0x7f0cd413c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 10 monclient(hunting): picked mon.noname-b con 0x7f0cd40b3d60 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd406c350 con 0x7f0cd40b00c0 2023-10-06T13:53:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd413cd50 con 0x7f0cd40b3d60 2023-10-06T13:53:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd4144b90 con 0x7f0cd40b3990 2023-10-06T13:53:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.836+0000 7f0cdbea4640 10 monclient(hunting): authenticate will time out at 1328.022805s 2023-10-06T13:53:19.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.840+0000 7f8b9f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ba00ae860 0x7f8ba00b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.841+0000 7f8b9f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ba00ae860 0x7f8ba00b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38936/0 (socket says 172.21.15.19:38936) 2023-10-06T13:53:19.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.841+0000 7f8b9f7fe640 1 -- 172.21.15.19:0/2916523069 learned_addr learned my addr 172.21.15.19:0/2916523069 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.841+0000 7f8b9f7fe640 10 monclient(hunting): get_auth_request con 0x7f8ba00ae860 auth_method 0 2023-10-06T13:53:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.841+0000 7f8b9f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.841+0000 7f8b9f7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.841+0000 7f8b9f7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7f8b9f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7f8b9f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7f8b9f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1b8ff9640 1 -- 172.21.15.19:0/2118078786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b403e450 con 0x7fc1bc0b3d60 2023-10-06T13:53:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1b8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1b8ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1b8ff9640 10 monclient: dump: 2023-10-06T13:53:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.941 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1b8ff9640 10 monclient: _finish_auth 0 2023-10-06T13:53:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1b8ff9640 10 monclient: _check_auth_tickets 2023-10-06T13:53:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1b8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1b8ff9640 1 -- 172.21.15.19:0/2118078786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b403e5d0 con 0x7fc1bc0b3d60 2023-10-06T13:53:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1c27e4640 5 monclient: authenticate success, global_id 4311 2023-10-06T13:53:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1c27e4640 10 monclient: _renew_subs 2023-10-06T13:53:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1c27e4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1c27e4640 1 -- 172.21.15.19:0/2118078786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc13dc20 con 0x7fc1bc0b3d60 2023-10-06T13:53:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1b8ff9640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1b8ff9640 1 -- 172.21.15.19:0/2118078786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b403e980 con 0x7fc1bc0b3d60 2023-10-06T13:53:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1b8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1b8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1b8ff9640 10 monclient: dump: 2023-10-06T13:53:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:19.943 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1c27e4640 10 client.4311.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:19.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1ba7fc640 4 set_mon_vals no callback set 2023-10-06T13:53:19.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1c27e4640 10 monclient: _renew_subs 2023-10-06T13:53:19.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1c27e4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1c27e4640 1 -- 172.21.15.19:0/2118078786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc137830 con 0x7fc1bc0b3d60 2023-10-06T13:53:19.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.842+0000 7fc1c27e4640 1 librados: init done 2023-10-06T13:53:19.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.843+0000 7f0cd9c40640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cd40b00c0 0x7f0cd4143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:19.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.843+0000 7fc1c27e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.844+0000 7fc1c27e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:19.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.844+0000 7fc1c27e4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.844+0000 7fc1c27e4640 1 -- 172.21.15.19:0/2118078786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1bc13de60 con 0x7fc1bc0b3d60 2023-10-06T13:53:19.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.847+0000 7f8b9f7fe640 10 monclient(hunting): handle_auth_done global_id 4314 payload 824 2023-10-06T13:53:19.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.847+0000 7f8b9f7fe640 1 -- 172.21.15.19:0/2916523069 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8ba00ad810 msgr2=0x7f8ba00a89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.847+0000 7f8b9f7fe640 1 --2- 172.21.15.19:0/2916523069 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8ba00ad810 0x7f8ba00a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.847+0000 7f8b9f7fe640 1 -- 172.21.15.19:0/2916523069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ba00aa8d0 msgr2=0x7f8ba00b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.847+0000 7f8b9f7fe640 1 --2- 172.21.15.19:0/2916523069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ba00aa8d0 0x7f8ba00b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.847+0000 7f8b9f7fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.847+0000 7f8b9f7fe640 1 monclient: found mon.noname-a 2023-10-06T13:53:19.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.847+0000 7f8b9f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.847+0000 7f8b9f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.947 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.847+0000 7f8b9f7fe640 1 -- 172.21.15.19:0/2916523069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b98009a70 con 0x7f8ba00ae860 2023-10-06T13:53:19.947 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.848+0000 7f0cd9c40640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cd40b00c0 0x7f0cd4143fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38944/0 (socket says 172.21.15.19:38944) 2023-10-06T13:53:19.947 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.848+0000 7f0cd9c40640 1 -- 172.21.15.19:0/3814081660 learned_addr learned my addr 172.21.15.19:0/3814081660 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:19.947 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.848+0000 7f0cd9c40640 10 monclient(hunting): get_auth_request con 0x7f0cd40b00c0 auth_method 0 2023-10-06T13:53:19.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.848+0000 7f0cd9c40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:19.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.848+0000 7f0cd9c40640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:19.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.848+0000 7f0cd9c40640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:19.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.852+0000 7f0cd9c40640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:19.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.852+0000 7f0cd9c40640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:19.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.852+0000 7f0cd9c40640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:19.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.856+0000 7f0cd9c40640 10 monclient(hunting): handle_auth_done global_id 4317 payload 824 2023-10-06T13:53:19.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.856+0000 7f0cd9c40640 1 -- 172.21.15.19:0/3814081660 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0cd40b3990 msgr2=0x7f0cd41ddb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.856+0000 7f0cd9c40640 1 --2- 172.21.15.19:0/3814081660 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0cd40b3990 0x7f0cd41ddb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.856+0000 7f0cd9c40640 1 -- 172.21.15.19:0/3814081660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cd40b3d60 msgr2=0x7f0cd413c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:19.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.856+0000 7f0cd9c40640 1 --2- 172.21.15.19:0/3814081660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cd40b3d60 0x7f0cd413c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.856+0000 7f0cd9c40640 10 monclient: _finish_hunting 0 2023-10-06T13:53:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.856+0000 7f0cd9c40640 1 monclient: found mon.noname-a 2023-10-06T13:53:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.856+0000 7f0cd9c40640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.856+0000 7f0cd9c40640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.856+0000 7f0cd9c40640 1 -- 172.21.15.19:0/3814081660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc0094d0 con 0x7f0cd40b00c0 2023-10-06T13:53:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.857+0000 7f8b9f7fe640 1 --2- 172.21.15.19:0/2916523069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ba00ae860 0x7f8ba00b41b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009e80 tx=0x7f8b98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.858+0000 7f8b87fff640 10 client.?.objecter ms_handle_connect 0x7f8ba00ae860 2023-10-06T13:53:19.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.858+0000 7f8b87fff640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.862+0000 7f0cd9c40640 1 --2- 172.21.15.19:0/3814081660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cd40b00c0 0x7f0cd4143fe0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000ba50 tx=0x7f0cd000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:19.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.862+0000 7f0cca7fc640 10 client.?.objecter ms_handle_connect 0x7f0cd40b00c0 2023-10-06T13:53:19.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.862+0000 7f0cca7fc640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:19.951 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.869+0000 7f0b2e050640 10 monclient: get_monmap_and_config 2023-10-06T13:53:19.951 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.869+0000 7f0b2e050640 10 monclient: build_initial_monmap 2023-10-06T13:53:19.951 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.869+0000 7f0b2e050640 10 monclient: monmap: 2023-10-06T13:53:19.951 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:19.870434+0000 2023-10-06T13:53:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:19.870434+0000 2023-10-06T13:53:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:19.952 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 1 Processor -- start 2023-10-06T13:53:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 1 -- start start 2023-10-06T13:53:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 10 monclient: init 2023-10-06T13:53:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 10 monclient: _add_conns ranks=[3,4,1,0,2] 2023-10-06T13:53:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b2806b6d0 0x7f0b28144200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 10 monclient(hunting): picked mon.noname-a con 0x7f0b2806b6d0 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0b28144740 0x7f0b2813df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 10 monclient(hunting): picked mon.noname-c con 0x7f0b28144740 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0b2813e460 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2806baa0 con 0x7f0b2813e460 2023-10-06T13:53:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b280b2530 con 0x7f0b2806b6d0 2023-10-06T13:53:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b280b2810 con 0x7f0b28144740 2023-10-06T13:53:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:19.870+0000 7f0b2e050640 10 monclient(hunting): authenticate will time out at 1328.057233s 2023-10-06T13:53:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.026+0000 7f123effd640 1 -- 172.21.15.19:0/711642149 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f1240013470 con 0x7f12500b3990 2023-10-06T13:53:20.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.027+0000 7f123effd640 1 --2- 172.21.15.19:0/711642149 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f123006c1c0 0x7f123006e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:20.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.027+0000 7f124dd9b640 1 --2- 172.21.15.19:0/711642149 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f123006c1c0 0x7f123006e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:20.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.027+0000 7f124dd9b640 10 monclient: get_auth_request con 0x7f123006c1c0 auth_method 0 2023-10-06T13:53:20.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.027+0000 7f124dd9b640 1 --2- 172.21.15.19:0/711642149 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f123006c1c0 0x7f123006e670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1244036000 tx=0x7f124403d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:20.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.027+0000 7f123effd640 10 client.4342.objecter ms_handle_connect 0x7f123006c1c0 2023-10-06T13:53:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.029+0000 7f123effd640 1 -- 172.21.15.19:0/711642149 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f12400588b0 con 0x7f12500b3990 2023-10-06T13:53:20.031 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.029+0000 7f123effd640 10 client.4342.objecter ms_dispatch 0x7f1250000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:20.031 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.029+0000 7f123effd640 3 client.4342.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:20.031 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.029+0000 7f123effd640 3 client.4342.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:20.031 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.029+0000 7f123effd640 20 client.4342.objecter dump_active .. 0 homeless 2023-10-06T13:53:20.042 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.040+0000 7f123effd640 1 -- 172.21.15.19:0/711642149 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f1240054bf0 con 0x7f12500b3990 2023-10-06T13:53:20.042 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.040+0000 7f123effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.043 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.040+0000 7f123effd640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.047+0000 7f0b26ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0b28144740 0x7f0b2813df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:20.075 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.074+0000 7f5bd1ffb640 1 -- 172.21.15.19:0/1355777657 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5bcc01b3b0 con 0x7f5bdc0b37e0 2023-10-06T13:53:20.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.074+0000 7f5bd1ffb640 1 --2- 172.21.15.19:0/1355777657 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5bc006c240 0x7f5bc006e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:20.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.074+0000 7f5be1606640 1 --2- 172.21.15.19:0/1355777657 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5bc006c240 0x7f5bc006e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:20.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.074+0000 7f5be1606640 10 monclient: get_auth_request con 0x7f5bc006c240 auth_method 0 2023-10-06T13:53:20.077 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.075+0000 7f5be1606640 1 --2- 172.21.15.19:0/1355777657 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5bc006c240 0x7f5bc006e6f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8036000 tx=0x7f5bd803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:20.077 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.075+0000 7f5bd1ffb640 10 client.4265.objecter ms_handle_connect 0x7f5bc006c240 2023-10-06T13:53:20.077 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cca7fc640 1 -- 172.21.15.19:0/3814081660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd000cbe0 con 0x7f0cd40b00c0 2023-10-06T13:53:20.077 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:20.077 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cca7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:20.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8b87fff640 1 -- 172.21.15.19:0/2916523069 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b98038440 con 0x7f8ba00ae860 2023-10-06T13:53:20.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8b87fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:20.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cca7fc640 10 monclient: dump: 2023-10-06T13:53:20.078 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:20.078 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:20.078 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.078 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.079 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:20.079 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:20.079 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:20.079 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:20.079 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:20.079 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:20.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cca7fc640 10 monclient: _finish_auth 0 2023-10-06T13:53:20.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cca7fc640 10 monclient: _check_auth_tickets 2023-10-06T13:53:20.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:20.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cdbea4640 5 monclient: authenticate success, global_id 4317 2023-10-06T13:53:20.080 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8b87fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:20.080 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8b87fff640 10 monclient: dump: 2023-10-06T13:53:20.080 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:20.080 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:20.080 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.080 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.080 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:20.080 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:20.080 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:20.080 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:20.080 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:20.080 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:20.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8b87fff640 10 monclient: _finish_auth 0 2023-10-06T13:53:20.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8b87fff640 10 monclient: _check_auth_tickets 2023-10-06T13:53:20.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8b87fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:20.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cdbea4640 10 monclient: _renew_subs 2023-10-06T13:53:20.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cdbea4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:20.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cdbea4640 1 -- 172.21.15.19:0/3814081660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd413d5c0 con 0x7f0cd40b00c0 2023-10-06T13:53:20.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8ba588f640 5 monclient: authenticate success, global_id 4314 2023-10-06T13:53:20.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8ba588f640 10 monclient: _renew_subs 2023-10-06T13:53:20.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8ba588f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:20.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8ba588f640 1 -- 172.21.15.19:0/2916523069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba013c7d0 con 0x7f8ba00ae860 2023-10-06T13:53:20.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cdbea4640 10 client.4317.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:20.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cdbea4640 10 monclient: _renew_subs 2023-10-06T13:53:20.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cdbea4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:20.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8ba588f640 10 client.4314.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:20.083 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8ba588f640 10 monclient: _renew_subs 2023-10-06T13:53:20.083 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8ba588f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:20.083 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cdbea4640 1 -- 172.21.15.19:0/3814081660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd413daf0 con 0x7f0cd40b00c0 2023-10-06T13:53:20.083 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8ba588f640 1 -- 172.21.15.19:0/2916523069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba00b48d0 con 0x7f8ba00ae860 2023-10-06T13:53:20.083 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f0cdbea4640 1 librados: init done 2023-10-06T13:53:20.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.076+0000 7f8ba588f640 1 librados: init done 2023-10-06T13:53:20.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.077+0000 7f0cdbea4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.077+0000 7f0cdbea4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.077+0000 7f0cdbea4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:20.085 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.077+0000 7f8ba588f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.085 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.077+0000 7f8ba588f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.085 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.077+0000 7f0cdbea4640 1 -- 172.21.15.19:0/3814081660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd413dd60 con 0x7f0cd40b00c0 2023-10-06T13:53:20.085 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.077+0000 7f8ba588f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:20.085 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.077+0000 7f8ba588f640 1 -- 172.21.15.19:0/2916523069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba00af7c0 con 0x7f8ba00ae860 2023-10-06T13:53:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.087+0000 7f5bd1ffb640 1 -- 172.21.15.19:0/1355777657 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5bcc0010c0 con 0x7f5bdc0b37e0 2023-10-06T13:53:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.087+0000 7f5bd1ffb640 10 client.4265.objecter ms_dispatch 0x7f5bdc000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.087+0000 7f5bd1ffb640 3 client.4265.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.087+0000 7f5bd1ffb640 3 client.4265.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.088+0000 7f5bd1ffb640 20 client.4265.objecter dump_active .. 0 homeless 2023-10-06T13:53:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.088+0000 7f5bd1ffb640 1 -- 172.21.15.19:0/1355777657 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f5bcc057520 con 0x7f5bdc0b37e0 2023-10-06T13:53:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.088+0000 7f5bd1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.088+0000 7f5bd1ffb640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78d57fa640 1 -- 172.21.15.19:0/765837724 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d003e610 con 0x7f78d80b3d60 2023-10-06T13:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78d57fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78d57fa640 10 monclient: dump: 2023-10-06T13:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:20.097 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78d57fa640 10 monclient: _finish_auth 0 2023-10-06T13:53:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78d57fa640 10 monclient: _check_auth_tickets 2023-10-06T13:53:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78d57fa640 1 -- 172.21.15.19:0/765837724 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d003e790 con 0x7f78d80b3d60 2023-10-06T13:53:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78def20640 5 monclient: authenticate success, global_id 4268 2023-10-06T13:53:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78def20640 10 monclient: _renew_subs 2023-10-06T13:53:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78def20640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78def20640 1 -- 172.21.15.19:0/765837724 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d806ce60 con 0x7f78d80b3d60 2023-10-06T13:53:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78d57fa640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78d57fa640 1 -- 172.21.15.19:0/765837724 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d003eb40 con 0x7f78d80b3d60 2023-10-06T13:53:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78d57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78d57fa640 10 monclient: dump: 2023-10-06T13:53:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78d6ffd640 4 set_mon_vals no callback set 2023-10-06T13:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78def20640 10 client.4268.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78def20640 10 monclient: _renew_subs 2023-10-06T13:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78def20640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78def20640 1 -- 172.21.15.19:0/765837724 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d81dadf0 con 0x7f78d80b3d60 2023-10-06T13:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.094+0000 7f78def20640 1 librados: init done 2023-10-06T13:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.095+0000 7f78def20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.095+0000 7f78def20640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.095+0000 7f78def20640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.095+0000 7f78def20640 1 -- 172.21.15.19:0/765837724 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d81db040 con 0x7f78d80b3d60 2023-10-06T13:53:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.133+0000 7f0cca7fc640 1 -- 172.21.15.19:0/3814081660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0009930 con 0x7f0cd40b00c0 2023-10-06T13:53:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.134+0000 7f0cca7fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.134+0000 7f0cca7fc640 1 -- 172.21.15.19:0/3814081660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd001e440 con 0x7f0cd40b00c0 2023-10-06T13:53:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.134+0000 7f0cca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.134+0000 7f0cca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.134+0000 7f0cca7fc640 10 monclient: dump: 2023-10-06T13:53:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:20.137 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.134+0000 7f0ccbfff640 4 set_mon_vals no callback set 2023-10-06T13:53:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.134+0000 7f8b87fff640 1 -- 172.21.15.19:0/2916523069 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b98038e90 con 0x7f8ba00ae860 2023-10-06T13:53:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.134+0000 7f8b87fff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.134+0000 7f8b87fff640 1 -- 172.21.15.19:0/2916523069 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b98009450 con 0x7f8ba00ae860 2023-10-06T13:53:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.134+0000 7f8b87fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.134+0000 7f8b87fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.134+0000 7f8b87fff640 10 monclient: dump: 2023-10-06T13:53:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:20.138 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.134+0000 7f8b9d7fa640 4 set_mon_vals no callback set 2023-10-06T13:53:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.136+0000 7f0b277fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b2806b6d0 0x7f0b28144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:20.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.177+0000 7f0b277fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b2806b6d0 0x7f0b28144200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38954/0 (socket says 172.21.15.19:38954) 2023-10-06T13:53:20.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.177+0000 7f0b277fe640 1 -- 172.21.15.19:0/273604029 learned_addr learned my addr 172.21.15.19:0/273604029 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:20.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.177+0000 7f0b277fe640 10 monclient(hunting): get_auth_request con 0x7f0b2806b6d0 auth_method 0 2023-10-06T13:53:20.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.177+0000 7f0b277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:20.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.177+0000 7f0b277fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:20.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.177+0000 7f0b277fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:20.268 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.266+0000 7f1254b3c640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2023-10-06T13:53:20.268 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.266+0000 7f1254b3c640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2023-10-06T13:53:20.268 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.266+0000 7f1254b3c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:20.268 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.266+0000 7f1254b3c640 1 -- 172.21.15.19:0/711642149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1250070030 con 0x7f12500b3990 2023-10-06T13:53:20.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.280+0000 7f0b26ffd640 10 monclient(hunting): get_auth_request con 0x7f0b28144740 auth_method 0 2023-10-06T13:53:20.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.280+0000 7f0b26ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:20.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.280+0000 7f0b26ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:20.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.280+0000 7f0b26ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:20.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.283+0000 7f0b26ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:20.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.283+0000 7f0b26ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:20.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.283+0000 7f0b26ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.286+0000 7f0b26ffd640 10 monclient(hunting): handle_auth_done global_id 4348 payload 274 2023-10-06T13:53:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.286+0000 7f0b26ffd640 1 -- 172.21.15.19:0/273604029 >> v2:172.21.15.19:3300/0 conn(0x7f0b2806b6d0 msgr2=0x7f0b28144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.286+0000 7f0b26ffd640 1 --2- 172.21.15.19:0/273604029 >> v2:172.21.15.19:3300/0 conn(0x7f0b2806b6d0 0x7f0b28144200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.286+0000 7f0b26ffd640 1 -- 172.21.15.19:0/273604029 >> v1:172.21.15.83:6789/0 conn(0x7f0b2813e460 legacy=0x7f0b280b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.286+0000 7f0b26ffd640 10 monclient: _finish_hunting 0 2023-10-06T13:53:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.286+0000 7f0b26ffd640 1 monclient: found mon.noname-c 2023-10-06T13:53:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.286+0000 7f0b26ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.286+0000 7f0b26ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:53:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.286+0000 7f0b26ffd640 1 -- 172.21.15.19:0/273604029 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b280aeff0 con 0x7f0b28144740 2023-10-06T13:53:20.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.289+0000 7f5be386a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2023-10-06T13:53:20.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.289+0000 7f5be386a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2023-10-06T13:53:20.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.289+0000 7f5be386a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:20.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.289+0000 7f5be386a640 1 -- 172.21.15.19:0/1355777657 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5bdc1dd620 con 0x7f5bdc0b37e0 2023-10-06T13:53:20.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.303+0000 7f0b26ffd640 1 --2- 172.21.15.19:0/273604029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b28144740 0x7f0b2813df20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0b280ae570 tx=0x7f0b1c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5fec219d8fae5ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:20.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.303+0000 7f23caffd640 1 -- 172.21.15.19:0/3700926455 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f23bc03f8f0 con 0x7f23cc0b3990 2023-10-06T13:53:20.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.304+0000 7f23caffd640 1 --2- 172.21.15.19:0/3700926455 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f23b806c1c0 0x7f23b806e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:20.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.304+0000 7f23d24bc640 1 --2- 172.21.15.19:0/3700926455 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f23b806c1c0 0x7f23b806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:20.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.304+0000 7f23d24bc640 10 monclient: get_auth_request con 0x7f23b806c1c0 auth_method 0 2023-10-06T13:53:20.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.304+0000 7f23d24bc640 1 --2- 172.21.15.19:0/3700926455 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f23b806c1c0 0x7f23b806e670 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f23c0007fd0 tx=0x7f23c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:20.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.304+0000 7f23caffd640 10 client.4345.objecter ms_handle_connect 0x7f23b806c1c0 2023-10-06T13:53:20.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.308+0000 7f23caffd640 1 -- 172.21.15.19:0/3700926455 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f23bc07b5d0 con 0x7f23cc0b3990 2023-10-06T13:53:20.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.308+0000 7f23caffd640 10 client.4345.objecter ms_dispatch 0x7f23cc000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:20.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.308+0000 7f23caffd640 3 client.4345.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:20.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.308+0000 7f23caffd640 3 client.4345.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:20.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.308+0000 7f23caffd640 20 client.4345.objecter dump_active .. 0 homeless 2023-10-06T13:53:20.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.322+0000 7f23caffd640 1 -- 172.21.15.19:0/3700926455 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f23bc077210 con 0x7f23cc0b3990 2023-10-06T13:53:20.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.322+0000 7f23caffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.322+0000 7f23caffd640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:20.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b267fc640 1 -- 172.21.15.19:0/273604029 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c031830 con 0x7f0b28144740 2023-10-06T13:53:20.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:20.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b267fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:20.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b267fc640 10 monclient: dump: 2023-10-06T13:53:20.330 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:20.331 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b267fc640 10 monclient: _finish_auth 0 2023-10-06T13:53:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b267fc640 10 monclient: _check_auth_tickets 2023-10-06T13:53:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b267fc640 1 -- 172.21.15.19:0/273604029 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c031990 con 0x7f0b28144740 2023-10-06T13:53:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b2e050640 5 monclient: authenticate success, global_id 4348 2023-10-06T13:53:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b2e050640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b267fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b2e050640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b2e050640 4 set_mon_vals no callback set 2023-10-06T13:53:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b267fc640 1 -- 172.21.15.19:0/273604029 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c031d20 con 0x7f0b28144740 2023-10-06T13:53:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b267fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f0b267fc640 10 monclient: dump: 2023-10-06T13:53:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:20.333 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:20.334 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f123effd640 1 -- 172.21.15.19:0/711642149 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1240021090 con 0x7f12500b3990 2023-10-06T13:53:20.334 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f123effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2023-10-06T13:53:20.334 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.328+0000 7f123effd640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:20.334 INFO:teuthology.orchestra.run.smithi019.stdout:64424509442 2023-10-06T13:53:20.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.329+0000 7f0b2e050640 10 monclient: shutdown 2023-10-06T13:53:20.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.329+0000 7f0b2e050640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:20.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.329+0000 7f0b2e050640 1 -- 172.21.15.19:0/273604029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b28144740 msgr2=0x7f0b2813df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:20.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.329+0000 7f0b2e050640 1 --2- 172.21.15.19:0/273604029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b28144740 0x7f0b2813df20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0b280ae570 tx=0x7f0b1c02b8d0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.329+0000 7f0b2e050640 1 -- 172.21.15.19:0/273604029 shutdown_connections 2023-10-06T13:53:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.329+0000 7f0b2e050640 1 --2- 172.21.15.19:0/273604029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b28144740 0x7f0b2813df20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.329+0000 7f0b2e050640 1 --2- 172.21.15.19:0/273604029 >> v2:172.21.15.19:3300/0 conn(0x7f0b2806b6d0 0x7f0b28144200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.329+0000 7f0b2e050640 1 -- 172.21.15.19:0/273604029 >> 172.21.15.19:0/273604029 conn(0x7f0b280665b0 msgr2=0x7f0b2806ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.329+0000 7f0b2e050640 1 -- 172.21.15.19:0/273604029 shutdown_connections 2023-10-06T13:53:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.329+0000 7f0b2e050640 1 -- 172.21.15.19:0/273604029 wait complete. 2023-10-06T13:53:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.330+0000 7f0b2e050640 10 monclient: build_initial_monmap 2023-10-06T13:53:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.330+0000 7f0b2e050640 10 monclient: monmap: 2023-10-06T13:53:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:20.337 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f0b2e050640 1 librados: starting msgr at 2023-10-06T13:53:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f0b2e050640 1 librados: starting objecter 2023-10-06T13:53:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f0b2e050640 1 Processor -- start 2023-10-06T13:53:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 10 librados: watch_flush enter 2023-10-06T13:53:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 10 librados: watch_flush exit 2023-10-06T13:53:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 20 client.4342.objecter shutdown clearing up homeless session... 2023-10-06T13:53:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 10 client.4342.objecter successfully canceled tick 2023-10-06T13:53:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f0b2e050640 1 -- start start 2023-10-06T13:53:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f0b2e050640 1 librados: setting wanted keys 2023-10-06T13:53:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f0b2e050640 1 librados: calling monclient init 2023-10-06T13:53:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f0b2e050640 10 monclient: init 2023-10-06T13:53:20.339 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 1 -- 172.21.15.19:0/711642149 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f123006c1c0 msgr2=0x7f123006e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:20.339 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 1 --2- 172.21.15.19:0/711642149 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f123006c1c0 0x7f123006e670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1244036000 tx=0x7f124403d040 comp rx=0 tx=0).stop 2023-10-06T13:53:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 10 monclient: shutdown 2023-10-06T13:53:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 1 -- 172.21.15.19:0/711642149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12500b3990 msgr2=0x7f12501da940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 1 --2- 172.21.15.19:0/711642149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12500b3990 0x7f12501da940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f124000e890 tx=0x7f124000ed50 comp rx=0 tx=0).stop 2023-10-06T13:53:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 1 -- 172.21.15.19:0/711642149 shutdown_connections 2023-10-06T13:53:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 1 --2- 172.21.15.19:0/711642149 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f123006c1c0 0x7f123006e670 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 1 --2- 172.21.15.19:0/711642149 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f1250141a40 0x7f12501dd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 1 --2- 172.21.15.19:0/711642149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12500b3d60 0x7f125006f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 1 --2- 172.21.15.19:0/711642149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12500b3990 0x7f12501da940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 1 -- 172.21.15.19:0/711642149 >> 172.21.15.19:0/711642149 conn(0x7f12500b6e00 msgr2=0x7f1250068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 1 -- 172.21.15.19:0/711642149 shutdown_connections 2023-10-06T13:53:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 1 -- 172.21.15.19:0/711642149 wait complete. 2023-10-06T13:53:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.331+0000 7f1254b3c640 1 librados: shutdown 2023-10-06T13:53:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T13:53:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b2806b6d0 0x7f0b280ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 10 monclient(hunting): picked mon.noname-a con 0x7f0b2806b6d0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2813e460 0x7f0b28135de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 10 monclient(hunting): picked mon.noname-b con 0x7f0b2813e460 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0b28144740 0x7f0b28136ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 10 monclient(hunting): picked mon.noname-c con 0x7f0b28144740 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b28140db0 con 0x7f0b2806b6d0 2023-10-06T13:53:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b281365b0 con 0x7f0b2813e460 2023-10-06T13:53:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b28141980 con 0x7f0b28144740 2023-10-06T13:53:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.332+0000 7f0b2e050640 10 monclient(hunting): authenticate will time out at 1328.518987s 2023-10-06T13:53:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.333+0000 7f0b26ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2813e460 0x7f0b28135de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.334+0000 7f0b26ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2813e460 0x7f0b28135de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39504/0 (socket says 172.21.15.19:39504) 2023-10-06T13:53:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.334+0000 7f0b26ffd640 1 -- 172.21.15.19:0/1027711831 learned_addr learned my addr 172.21.15.19:0/1027711831 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.334+0000 7f0b26ffd640 10 monclient(hunting): get_auth_request con 0x7f0b2813e460 auth_method 0 2023-10-06T13:53:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.334+0000 7f0b26ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.334+0000 7f0b26ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.334+0000 7f0b26ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.334+0000 7f0b26ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.334+0000 7f0b26ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.334+0000 7f0b26ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.337+0000 7f0b26ffd640 10 monclient(hunting): handle_auth_done global_id 4351 payload 824 2023-10-06T13:53:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.337+0000 7f0b26ffd640 1 -- 172.21.15.19:0/1027711831 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0b28144740 msgr2=0x7f0b28136ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.337+0000 7f0b26ffd640 1 --2- 172.21.15.19:0/1027711831 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0b28144740 0x7f0b28136ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.337+0000 7f0b26ffd640 1 -- 172.21.15.19:0/1027711831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b2806b6d0 msgr2=0x7f0b280ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.337+0000 7f0b26ffd640 1 --2- 172.21.15.19:0/1027711831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b2806b6d0 0x7f0b280ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.337+0000 7f0b26ffd640 10 monclient: _finish_hunting 0 2023-10-06T13:53:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.337+0000 7f0b26ffd640 1 monclient: found mon.noname-b 2023-10-06T13:53:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.337+0000 7f0b26ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.337+0000 7f0b26ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:20.346 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.337+0000 7f0b26ffd640 1 -- 172.21.15.19:0/1027711831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c009400 con 0x7f0b2813e460 2023-10-06T13:53:20.346 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.339+0000 7f0b26ffd640 1 --2- 172.21.15.19:0/1027711831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2813e460 0x7f0b28135de0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c000c00 tx=0x7f0b1c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:20.346 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.339+0000 7f0b0ffff640 10 client.?.objecter ms_handle_connect 0x7f0b2813e460 2023-10-06T13:53:20.346 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.339+0000 7f0b0ffff640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:20.347 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.343+0000 7f0b0ffff640 1 -- 172.21.15.19:0/1027711831 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c03a960 con 0x7f0b2813e460 2023-10-06T13:53:20.347 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.343+0000 7f0b0ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:20.347 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.343+0000 7f0b0ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:20.347 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.343+0000 7f0b0ffff640 10 monclient: dump: 2023-10-06T13:53:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:20.348 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.343+0000 7f0b0ffff640 10 monclient: _finish_auth 0 2023-10-06T13:53:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.343+0000 7f0b0ffff640 10 monclient: _check_auth_tickets 2023-10-06T13:53:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.343+0000 7f0b0ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.343+0000 7f0b0ffff640 1 -- 172.21.15.19:0/1027711831 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c03aac0 con 0x7f0b2813e460 2023-10-06T13:53:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.343+0000 7f0b2e050640 5 monclient: authenticate success, global_id 4351 2023-10-06T13:53:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.343+0000 7f0b0ffff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.343+0000 7f0b25ffb640 4 set_mon_vals no callback set 2023-10-06T13:53:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.344+0000 7f0b2e050640 10 monclient: _renew_subs 2023-10-06T13:53:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.344+0000 7f0b0ffff640 1 -- 172.21.15.19:0/1027711831 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c03ae30 con 0x7f0b2813e460 2023-10-06T13:53:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.344+0000 7f0b2e050640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.344+0000 7f0b2e050640 1 -- 172.21.15.19:0/1027711831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b28138370 con 0x7f0b2813e460 2023-10-06T13:53:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.344+0000 7f0b0ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.344+0000 7f0b0ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.344+0000 7f0b0ffff640 10 monclient: dump: 2023-10-06T13:53:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:20.350 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.344+0000 7f0b2e050640 10 client.4351.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.344+0000 7f0b2e050640 10 monclient: _renew_subs 2023-10-06T13:53:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.344+0000 7f0b2e050640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.344+0000 7f0b2e050640 1 -- 172.21.15.19:0/1027711831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b28138880 con 0x7f0b2813e460 2023-10-06T13:53:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.344+0000 7f0b2e050640 1 librados: init done 2023-10-06T13:53:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.345+0000 7f0b2e050640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.345+0000 7f0b2e050640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.345+0000 7f0b2e050640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.345+0000 7f0b2e050640 1 -- 172.21.15.19:0/1027711831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b2804e000 con 0x7f0b2813e460 2023-10-06T13:53:20.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509442 for osd.2 2023-10-06T13:53:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.358+0000 7f5bd1ffb640 1 -- 172.21.15.19:0/1355777657 <== mon.2 v2:172.21.15.19:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5bcc017090 con 0x7f5bdc0b37e0 2023-10-06T13:53:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.358+0000 7f5bd1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2023-10-06T13:53:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.358+0000 7f5bd1ffb640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:20.360 INFO:teuthology.orchestra.run.smithi019.stdout:68719476741 2023-10-06T13:53:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 10 librados: watch_flush enter 2023-10-06T13:53:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 10 librados: watch_flush exit 2023-10-06T13:53:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 20 client.4265.objecter shutdown clearing up homeless session... 2023-10-06T13:53:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 10 client.4265.objecter successfully canceled tick 2023-10-06T13:53:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 1 -- 172.21.15.19:0/1355777657 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5bc006c240 msgr2=0x7f5bc006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 1 --2- 172.21.15.19:0/1355777657 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5bc006c240 0x7f5bc006e6f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8036000 tx=0x7f5bd803c040 comp rx=0 tx=0).stop 2023-10-06T13:53:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 10 monclient: shutdown 2023-10-06T13:53:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 1 -- 172.21.15.19:0/1355777657 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bdc0b37e0 msgr2=0x7f5bdc1274c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 1 --2- 172.21.15.19:0/1355777657 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bdc0b37e0 0x7f5bdc1274c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c440 tx=0x7f5bcc00c900 comp rx=0 tx=0).stop 2023-10-06T13:53:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 1 -- 172.21.15.19:0/1355777657 shutdown_connections 2023-10-06T13:53:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 1 --2- 172.21.15.19:0/1355777657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bdc145430 0x7f5bdc1dd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 1 --2- 172.21.15.19:0/1355777657 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5bc006c240 0x7f5bc006e6f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 1 --2- 172.21.15.19:0/1355777657 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5bdc0b37e0 0x7f5bdc1274c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 1 --2- 172.21.15.19:0/1355777657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bdc0b3410 0x7f5bdc0aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 1 -- 172.21.15.19:0/1355777657 >> 172.21.15.19:0/1355777657 conn(0x7f5bdc0b6e00 msgr2=0x7f5bdc06bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 1 -- 172.21.15.19:0/1355777657 shutdown_connections 2023-10-06T13:53:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 1 -- 172.21.15.19:0/1355777657 wait complete. 2023-10-06T13:53:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.361+0000 7f5be386a640 1 librados: shutdown 2023-10-06T13:53:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.363+0000 7f0b0ffff640 1 -- 172.21.15.19:0/1027711831 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0b1c04d450 con 0x7f0b2813e460 2023-10-06T13:53:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.364+0000 7f0b0ffff640 1 --2- 172.21.15.19:0/1027711831 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0b1406c1f0 0x7f0b1406e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.364+0000 7f0b277fe640 1 --2- 172.21.15.19:0/1027711831 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0b1406c1f0 0x7f0b1406e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.364+0000 7f0b277fe640 10 monclient: get_auth_request con 0x7f0b1406c1f0 auth_method 0 2023-10-06T13:53:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.364+0000 7f0b0ffff640 10 client.4351.objecter ms_handle_connect 0x7f0b1406c1f0 2023-10-06T13:53:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.364+0000 7f0b277fe640 1 --2- 172.21.15.19:0/1027711831 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0b1406c1f0 0x7f0b1406e6a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009770 tx=0x7f0b18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.366+0000 7f0b0ffff640 1 -- 172.21.15.19:0/1027711831 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0b1c07ae60 con 0x7f0b2813e460 2023-10-06T13:53:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.366+0000 7f0b0ffff640 10 client.4351.objecter ms_dispatch 0x7f0b28000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.366+0000 7f0b0ffff640 3 client.4351.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.366+0000 7f0b0ffff640 3 client.4351.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.366+0000 7f0b0ffff640 20 client.4351.objecter dump_active .. 0 homeless 2023-10-06T13:53:20.374 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.373+0000 7f78d57fa640 1 -- 172.21.15.19:0/765837724 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f78d003b050 con 0x7f78d80b3d60 2023-10-06T13:53:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.373+0000 7f78d57fa640 1 --2- 172.21.15.19:0/765837724 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f78bc06c1c0 0x7f78bc06e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.373+0000 7f78dccbc640 1 --2- 172.21.15.19:0/765837724 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f78bc06c1c0 0x7f78bc06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.373+0000 7f78dccbc640 10 monclient: get_auth_request con 0x7f78bc06c1c0 auth_method 0 2023-10-06T13:53:20.376 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.374+0000 7f78dccbc640 1 --2- 172.21.15.19:0/765837724 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f78bc06c1c0 0x7f78bc06e670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0011e0 tx=0x7f78cc007890 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:20.376 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.374+0000 7f78d57fa640 10 client.4268.objecter ms_handle_connect 0x7f78bc06c1c0 2023-10-06T13:53:20.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476741 for osd.1 2023-10-06T13:53:20.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.388+0000 7f78d57fa640 1 -- 172.21.15.19:0/765837724 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f78d007b2f0 con 0x7f78d80b3d60 2023-10-06T13:53:20.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.388+0000 7f78d57fa640 10 client.4268.objecter ms_dispatch 0x7f78d8000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:20.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.388+0000 7f78d57fa640 3 client.4268.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:20.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.388+0000 7f78d57fa640 3 client.4268.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:20.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.388+0000 7f78d57fa640 20 client.4268.objecter dump_active .. 0 homeless 2023-10-06T13:53:20.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.388+0000 7f78d57fa640 1 -- 172.21.15.19:0/765837724 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f78d0076f30 con 0x7f78d80b3d60 2023-10-06T13:53:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.388+0000 7f78d57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.388+0000 7f78d57fa640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:20.442 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.440+0000 7f8b87fff640 1 -- 172.21.15.19:0/2916523069 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8b98036980 con 0x7f8ba00ae860 2023-10-06T13:53:20.442 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.440+0000 7f8b87fff640 1 --2- 172.21.15.19:0/2916523069 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8b8c06c1c0 0x7f8b8c06e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.441+0000 7f8b9effd640 1 --2- 172.21.15.19:0/2916523069 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8b8c06c1c0 0x7f8b8c06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:20.445 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.441+0000 7f8b9effd640 10 monclient: get_auth_request con 0x7f8b8c06c1c0 auth_method 0 2023-10-06T13:53:20.445 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.441+0000 7f8b87fff640 10 client.4314.objecter ms_handle_connect 0x7f8b8c06c1c0 2023-10-06T13:53:20.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.441+0000 7f8b9effd640 1 --2- 172.21.15.19:0/2916523069 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8b8c06c1c0 0x7f8b8c06e670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8b90009710 tx=0x7f8b90006c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:20.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.520+0000 7f23d4720640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2023-10-06T13:53:20.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.520+0000 7f23d4720640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-10-06T13:53:20.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.520+0000 7f23d4720640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:20.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.520+0000 7f23d4720640 1 -- 172.21.15.19:0/3700926455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f23cc0ae200 con 0x7f23cc0b3990 2023-10-06T13:53:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.569+0000 7f78def20640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2023-10-06T13:53:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.569+0000 7f78def20640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2023-10-06T13:53:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.569+0000 7f78def20640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.569+0000 7f78def20640 1 -- 172.21.15.19:0/765837724 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f78d80b0cc0 con 0x7f78d80b3d60 2023-10-06T13:53:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.574+0000 7f78d57fa640 1 -- 172.21.15.19:0/765837724 <== mon.2 v2:172.21.15.19:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f78d004e020 con 0x7f78d80b3d60 2023-10-06T13:53:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.574+0000 7f78d57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2023-10-06T13:53:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.574+0000 7f78d57fa640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:20.576 INFO:teuthology.orchestra.run.smithi019.stdout:60129542149 2023-10-06T13:53:20.578 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 10 librados: watch_flush enter 2023-10-06T13:53:20.579 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 10 librados: watch_flush exit 2023-10-06T13:53:20.579 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 20 client.4268.objecter shutdown clearing up homeless session... 2023-10-06T13:53:20.579 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 10 client.4268.objecter successfully canceled tick 2023-10-06T13:53:20.579 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 1 -- 172.21.15.19:0/765837724 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f78bc06c1c0 msgr2=0x7f78bc06e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:20.579 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 1 --2- 172.21.15.19:0/765837724 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f78bc06c1c0 0x7f78bc06e670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0011e0 tx=0x7f78cc007890 comp rx=0 tx=0).stop 2023-10-06T13:53:20.579 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 10 monclient: shutdown 2023-10-06T13:53:20.579 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:20.579 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 1 -- 172.21.15.19:0/765837724 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f78d80b3d60 msgr2=0x7f78d81da6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:20.580 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 1 --2- 172.21.15.19:0/765837724 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f78d80b3d60 0x7f78d81da6d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009d90 tx=0x7f78d0004a40 comp rx=0 tx=0).stop 2023-10-06T13:53:20.580 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 1 -- 172.21.15.19:0/765837724 shutdown_connections 2023-10-06T13:53:20.580 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 1 --2- 172.21.15.19:0/765837724 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f78bc06c1c0 0x7f78bc06e670 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.580 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 1 --2- 172.21.15.19:0/765837724 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f78d80b3d60 0x7f78d81da6d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.580 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 1 --2- 172.21.15.19:0/765837724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78d80b3990 0x7f78d80a8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.580 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 1 --2- 172.21.15.19:0/765837724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78d80b00c0 0x7f78d806f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.580 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 1 -- 172.21.15.19:0/765837724 >> 172.21.15.19:0/765837724 conn(0x7f78d80b6e00 msgr2=0x7f78d80b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:20.580 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 1 -- 172.21.15.19:0/765837724 shutdown_connections 2023-10-06T13:53:20.580 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 1 -- 172.21.15.19:0/765837724 wait complete. 2023-10-06T13:53:20.580 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.576+0000 7f78def20640 1 librados: shutdown 2023-10-06T13:53:20.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2023-10-06T13:53:20.593 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.591+0000 7f0b0ffff640 1 -- 172.21.15.19:0/1027711831 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f0b1c076aa0 con 0x7f0b2813e460 2023-10-06T13:53:20.593 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.591+0000 7f0b0ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.593 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.591+0000 7f0b0ffff640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:20.598 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.597+0000 7f23caffd640 1 -- 172.21.15.19:0/3700926455 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f23bc04d020 con 0x7f23cc0b3990 2023-10-06T13:53:20.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.597+0000 7f23caffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-10-06T13:53:20.599 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.597+0000 7f23caffd640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:20.599 INFO:teuthology.orchestra.run.smithi019.stdout:64424509444 2023-10-06T13:53:20.602 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.599+0000 7f23d4720640 10 librados: watch_flush enter 2023-10-06T13:53:20.602 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.599+0000 7f23d4720640 10 librados: watch_flush exit 2023-10-06T13:53:20.602 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.599+0000 7f23d4720640 20 client.4345.objecter shutdown clearing up homeless session... 2023-10-06T13:53:20.602 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.599+0000 7f23d4720640 10 client.4345.objecter successfully canceled tick 2023-10-06T13:53:20.602 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.599+0000 7f23d4720640 1 -- 172.21.15.19:0/3700926455 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f23b806c1c0 msgr2=0x7f23b806e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:20.602 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.599+0000 7f23d4720640 1 --2- 172.21.15.19:0/3700926455 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f23b806c1c0 0x7f23b806e670 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f23c0007fd0 tx=0x7f23c0009040 comp rx=0 tx=0).stop 2023-10-06T13:53:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.599+0000 7f23d4720640 10 monclient: shutdown 2023-10-06T13:53:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.599+0000 7f23d4720640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.599+0000 7f23d4720640 1 -- 172.21.15.19:0/3700926455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23cc0b3990 msgr2=0x7f23cc0abe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.599+0000 7f23d4720640 1 --2- 172.21.15.19:0/3700926455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23cc0b3990 0x7f23cc0abe70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009d90 tx=0x7f23bc004a40 comp rx=0 tx=0).stop 2023-10-06T13:53:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.600+0000 7f23d4720640 1 -- 172.21.15.19:0/3700926455 shutdown_connections 2023-10-06T13:53:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.600+0000 7f23d4720640 1 --2- 172.21.15.19:0/3700926455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23cc0b3d60 0x7f23cc1d9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.600+0000 7f23d4720640 1 --2- 172.21.15.19:0/3700926455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23cc0b3990 0x7f23cc0abe70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.600+0000 7f23d4720640 1 --2- 172.21.15.19:0/3700926455 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f23b806c1c0 0x7f23b806e670 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.600+0000 7f23d4720640 1 --2- 172.21.15.19:0/3700926455 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23cc0b00c0 0x7f23cc1dd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.600+0000 7f23d4720640 1 -- 172.21.15.19:0/3700926455 >> 172.21.15.19:0/3700926455 conn(0x7f23cc0b6e00 msgr2=0x7f23cc0b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:20.604 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.600+0000 7f23d4720640 1 -- 172.21.15.19:0/3700926455 shutdown_connections 2023-10-06T13:53:20.604 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.600+0000 7f23d4720640 1 -- 172.21.15.19:0/3700926455 wait complete. 2023-10-06T13:53:20.604 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.600+0000 7f23d4720640 1 librados: shutdown 2023-10-06T13:53:20.616 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.4 2023-10-06T13:53:20.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.713+0000 7fc1b8ff9640 1 -- 172.21.15.19:0/2118078786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc1b403eae0 con 0x7fc1bc0b3d60 2023-10-06T13:53:20.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.713+0000 7fc1b8ff9640 1 --2- 172.21.15.19:0/2118078786 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc1a806c1f0 0x7fc1a806e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:20.716 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.713+0000 7fc1bbfff640 1 --2- 172.21.15.19:0/2118078786 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc1a806c1f0 0x7fc1a806e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:20.716 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.713+0000 7fc1bbfff640 10 monclient: get_auth_request con 0x7fc1a806c1f0 auth_method 0 2023-10-06T13:53:20.716 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.714+0000 7fc1b8ff9640 10 client.4311.objecter ms_handle_connect 0x7fc1a806c1f0 2023-10-06T13:53:20.716 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.714+0000 7fc1bbfff640 1 --2- 172.21.15.19:0/2118078786 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc1a806c1f0 0x7fc1a806e6a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009710 tx=0x7fc1b0006c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.728+0000 7f8b87fff640 1 -- 172.21.15.19:0/2916523069 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8b9807f8b0 con 0x7f8ba00ae860 2023-10-06T13:53:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.728+0000 7f8b87fff640 10 client.4314.objecter ms_dispatch 0x7f8ba0000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.728+0000 7f8b87fff640 3 client.4314.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.728+0000 7f8b87fff640 3 client.4314.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.728+0000 7f8b87fff640 20 client.4314.objecter dump_active .. 0 homeless 2023-10-06T13:53:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.728+0000 7f8b87fff640 1 -- 172.21.15.19:0/2916523069 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f8b9807b4f0 con 0x7f8ba00ae860 2023-10-06T13:53:20.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.728+0000 7f8b87fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.728+0000 7f8b87fff640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:20.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.745+0000 7fc1b8ff9640 1 -- 172.21.15.19:0/2118078786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc1b407bb00 con 0x7fc1bc0b3d60 2023-10-06T13:53:20.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.745+0000 7fc1b8ff9640 10 client.4311.objecter ms_dispatch 0x7fc1bc000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:20.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.745+0000 7fc1b8ff9640 3 client.4311.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:20.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.745+0000 7fc1b8ff9640 3 client.4311.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:20.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.745+0000 7fc1b8ff9640 20 client.4311.objecter dump_active .. 0 homeless 2023-10-06T13:53:20.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.745+0000 7fc1b8ff9640 1 -- 172.21.15.19:0/2118078786 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7fc1b4077740 con 0x7fc1bc0b3d60 2023-10-06T13:53:20.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.745+0000 7fc1b8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.745+0000 7fc1b8ff9640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:20.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.759+0000 7f0cca7fc640 1 -- 172.21.15.19:0/3814081660 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0cd001d3b0 con 0x7f0cd40b00c0 2023-10-06T13:53:20.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.760+0000 7f0cca7fc640 1 --2- 172.21.15.19:0/3814081660 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0cb806c1f0 0x7f0cb806e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:20.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.760+0000 7f0cd943f640 1 --2- 172.21.15.19:0/3814081660 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0cb806c1f0 0x7f0cb806e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:20.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.760+0000 7f0cd943f640 10 monclient: get_auth_request con 0x7f0cb806c1f0 auth_method 0 2023-10-06T13:53:20.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.760+0000 7f0cca7fc640 10 client.4317.objecter ms_handle_connect 0x7f0cb806c1f0 2023-10-06T13:53:20.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.760+0000 7f0cd943f640 1 --2- 172.21.15.19:0/3814081660 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0cb806c1f0 0x7f0cb806e6a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0cd413c6f0 tx=0x7f0cc4006ba0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.774+0000 7f0cca7fc640 1 -- 172.21.15.19:0/3814081660 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0cd0058320 con 0x7f0cd40b00c0 2023-10-06T13:53:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.774+0000 7f0cca7fc640 10 client.4317.objecter ms_dispatch 0x7f0cd4000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.774+0000 7f0cca7fc640 3 client.4317.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.774+0000 7f0cca7fc640 3 client.4317.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.774+0000 7f0cca7fc640 20 client.4317.objecter dump_active .. 0 homeless 2023-10-06T13:53:20.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.774+0000 7f0cca7fc640 1 -- 172.21.15.19:0/3814081660 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f0cd0054660 con 0x7f0cd40b00c0 2023-10-06T13:53:20.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.774+0000 7f0cca7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:20.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.774+0000 7f0cca7fc640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.786+0000 7f0b2e050640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2023-10-06T13:53:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.786+0000 7f0b2e050640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2023-10-06T13:53:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.786+0000 7f0b2e050640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.786+0000 7f0b2e050640 1 -- 172.21.15.19:0/1027711831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0b280af2f0 con 0x7f0b2813e460 2023-10-06T13:53:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.789+0000 7f0b0ffff640 1 -- 172.21.15.19:0/1027711831 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0b1c035070 con 0x7f0b2813e460 2023-10-06T13:53:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.789+0000 7f0b0ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2023-10-06T13:53:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.789+0000 7f0b0ffff640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:20.792 INFO:teuthology.orchestra.run.smithi019.stdout:60129542148 2023-10-06T13:53:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 10 librados: watch_flush enter 2023-10-06T13:53:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 10 librados: watch_flush exit 2023-10-06T13:53:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 20 client.4351.objecter shutdown clearing up homeless session... 2023-10-06T13:53:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 10 client.4351.objecter successfully canceled tick 2023-10-06T13:53:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 1 -- 172.21.15.19:0/1027711831 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0b1406c1f0 msgr2=0x7f0b1406e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 1 --2- 172.21.15.19:0/1027711831 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0b1406c1f0 0x7f0b1406e6a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009770 tx=0x7f0b18008040 comp rx=0 tx=0).stop 2023-10-06T13:53:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 10 monclient: shutdown 2023-10-06T13:53:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 1 -- 172.21.15.19:0/1027711831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2813e460 msgr2=0x7f0b28135de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 1 --2- 172.21.15.19:0/1027711831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2813e460 0x7f0b28135de0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c000c00 tx=0x7f0b1c0046c0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 1 -- 172.21.15.19:0/1027711831 shutdown_connections 2023-10-06T13:53:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 1 --2- 172.21.15.19:0/1027711831 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0b1406c1f0 0x7f0b1406e6a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 1 --2- 172.21.15.19:0/1027711831 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0b28144740 0x7f0b28136ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 1 --2- 172.21.15.19:0/1027711831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2813e460 0x7f0b28135de0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 1 --2- 172.21.15.19:0/1027711831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b2806b6d0 0x7f0b280ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:20.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 1 -- 172.21.15.19:0/1027711831 >> 172.21.15.19:0/1027711831 conn(0x7f0b280665b0 msgr2=0x7f0b280affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 1 -- 172.21.15.19:0/1027711831 shutdown_connections 2023-10-06T13:53:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 1 -- 172.21.15.19:0/1027711831 wait complete. 2023-10-06T13:53:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.792+0000 7f0b2e050640 1 librados: shutdown 2023-10-06T13:53:20.809 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2023-10-06T13:53:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.941+0000 7fc1c27e4640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2023-10-06T13:53:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.941+0000 7fc1c27e4640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2023-10-06T13:53:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.941+0000 7fc1c27e4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:20.941+0000 7fc1c27e4640 1 -- 172.21.15.19:0/2118078786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc1bc0a93b0 con 0x7fc1bc0b3d60 2023-10-06T13:53:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.000+0000 7f8ba588f640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2023-10-06T13:53:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.000+0000 7f8ba588f640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2023-10-06T13:53:21.003 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.000+0000 7f8ba588f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:21.003 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.000+0000 7f8ba588f640 1 -- 172.21.15.19:0/2916523069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8ba01409a0 con 0x7f8ba00ae860 2023-10-06T13:53:21.045 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.043+0000 7f0cdbea4640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2023-10-06T13:53:21.045 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.043+0000 7f0cdbea4640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2023-10-06T13:53:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.043+0000 7f0cdbea4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.043+0000 7f0cdbea4640 1 -- 172.21.15.19:0/3814081660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0cd413cf10 con 0x7f0cd40b00c0 2023-10-06T13:53:21.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.279+0000 7fc1b8ff9640 1 -- 172.21.15.19:0/2118078786 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc1bc0a93b0 con 0x7fc1bc0b3d60 2023-10-06T13:53:21.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.279+0000 7fc1b8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2023-10-06T13:53:21.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.279+0000 7fc1b8ff9640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:21.282 INFO:teuthology.orchestra.run.smithi019.stdout:55834574851 2023-10-06T13:53:21.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.280+0000 7f0cca7fc640 1 -- 172.21.15.19:0/3814081660 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0cd0019020 con 0x7f0cd40b00c0 2023-10-06T13:53:21.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.280+0000 7f0cca7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2023-10-06T13:53:21.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.280+0000 7f0cca7fc640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:21.283 INFO:teuthology.orchestra.run.smithi019.stdout:68719476741 2023-10-06T13:53:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.281+0000 7fc1c27e4640 10 librados: watch_flush enter 2023-10-06T13:53:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 10 librados: watch_flush exit 2023-10-06T13:53:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 20 client.4311.objecter shutdown clearing up homeless session... 2023-10-06T13:53:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 10 client.4311.objecter successfully canceled tick 2023-10-06T13:53:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 1 -- 172.21.15.19:0/2118078786 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc1a806c1f0 msgr2=0x7fc1a806e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 1 --2- 172.21.15.19:0/2118078786 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc1a806c1f0 0x7fc1a806e6a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009710 tx=0x7fc1b0006c90 comp rx=0 tx=0).stop 2023-10-06T13:53:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 10 monclient: shutdown 2023-10-06T13:53:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 1 -- 172.21.15.19:0/2118078786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1bc0b3d60 msgr2=0x7fc1bc137110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 1 --2- 172.21.15.19:0/2118078786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1bc0b3d60 0x7fc1bc137110 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4036040 tx=0x7fc1b4004a40 comp rx=0 tx=0).stop 2023-10-06T13:53:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 1 -- 172.21.15.19:0/2118078786 shutdown_connections 2023-10-06T13:53:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 1 --2- 172.21.15.19:0/2118078786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1bc0b3d60 0x7fc1bc137110 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 1 --2- 172.21.15.19:0/2118078786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1bc0b3990 0x7fc1bc13d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 1 --2- 172.21.15.19:0/2118078786 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc1a806c1f0 0x7fc1a806e6a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 1 --2- 172.21.15.19:0/2118078786 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc1bc0b00c0 0x7fc1bc144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 1 -- 172.21.15.19:0/2118078786 >> 172.21.15.19:0/2118078786 conn(0x7fc1bc0b6e00 msgr2=0x7fc1bc13a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 1 -- 172.21.15.19:0/2118078786 shutdown_connections 2023-10-06T13:53:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 10 librados: watch_flush enter 2023-10-06T13:53:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 10 librados: watch_flush exit 2023-10-06T13:53:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 20 client.4317.objecter shutdown clearing up homeless session... 2023-10-06T13:53:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 10 client.4317.objecter successfully canceled tick 2023-10-06T13:53:21.286 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 1 -- 172.21.15.19:0/2118078786 wait complete. 2023-10-06T13:53:21.286 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7fc1c27e4640 1 librados: shutdown 2023-10-06T13:53:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 1 -- 172.21.15.19:0/3814081660 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0cb806c1f0 msgr2=0x7f0cb806e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 1 --2- 172.21.15.19:0/3814081660 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0cb806c1f0 0x7f0cb806e6a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0cd413c6f0 tx=0x7f0cc4006ba0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 10 monclient: shutdown 2023-10-06T13:53:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 1 -- 172.21.15.19:0/3814081660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cd40b00c0 msgr2=0x7f0cd4143fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 1 --2- 172.21.15.19:0/3814081660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cd40b00c0 0x7f0cd4143fe0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000ba50 tx=0x7f0cd000bf10 comp rx=0 tx=0).stop 2023-10-06T13:53:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 1 -- 172.21.15.19:0/3814081660 shutdown_connections 2023-10-06T13:53:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 1 --2- 172.21.15.19:0/3814081660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cd40b3d60 0x7f0cd413c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 1 --2- 172.21.15.19:0/3814081660 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0cb806c1f0 0x7f0cb806e6a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 1 --2- 172.21.15.19:0/3814081660 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0cd40b3990 0x7f0cd41ddb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 1 --2- 172.21.15.19:0/3814081660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cd40b00c0 0x7f0cd4143fe0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 1 -- 172.21.15.19:0/3814081660 >> 172.21.15.19:0/3814081660 conn(0x7f0cd40b6e00 msgr2=0x7f0cd4139f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 1 -- 172.21.15.19:0/3814081660 shutdown_connections 2023-10-06T13:53:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 1 -- 172.21.15.19:0/3814081660 wait complete. 2023-10-06T13:53:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.282+0000 7f0cdbea4640 1 librados: shutdown 2023-10-06T13:53:21.296 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.7 2023-10-06T13:53:21.296 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476741 for osd.0 2023-10-06T13:53:21.333 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.331+0000 7f8b87fff640 1 -- 172.21.15.19:0/2916523069 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8b9803d020 con 0x7f8ba00ae860 2023-10-06T13:53:21.333 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.331+0000 7f8b87fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2023-10-06T13:53:21.333 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.331+0000 7f8b87fff640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:21.334 INFO:teuthology.orchestra.run.smithi019.stdout:64424509446 2023-10-06T13:53:21.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 10 librados: watch_flush enter 2023-10-06T13:53:21.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 10 librados: watch_flush exit 2023-10-06T13:53:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 20 client.4314.objecter shutdown clearing up homeless session... 2023-10-06T13:53:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 10 client.4314.objecter successfully canceled tick 2023-10-06T13:53:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 1 -- 172.21.15.19:0/2916523069 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8b8c06c1c0 msgr2=0x7f8b8c06e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 1 --2- 172.21.15.19:0/2916523069 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8b8c06c1c0 0x7f8b8c06e670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8b90009710 tx=0x7f8b90006c90 comp rx=0 tx=0).stop 2023-10-06T13:53:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 10 monclient: shutdown 2023-10-06T13:53:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 1 -- 172.21.15.19:0/2916523069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ba00ae860 msgr2=0x7f8ba00b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 1 --2- 172.21.15.19:0/2916523069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ba00ae860 0x7f8ba00b41b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009e80 tx=0x7f8b98004a40 comp rx=0 tx=0).stop 2023-10-06T13:53:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 1 -- 172.21.15.19:0/2916523069 shutdown_connections 2023-10-06T13:53:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 1 --2- 172.21.15.19:0/2916523069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ba00ae860 0x7f8ba00b41b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 1 --2- 172.21.15.19:0/2916523069 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8b8c06c1c0 0x7f8b8c06e670 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 1 --2- 172.21.15.19:0/2916523069 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8ba00ad810 0x7f8ba00a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.337 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 1 --2- 172.21.15.19:0/2916523069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ba00aa8d0 0x7f8ba00b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.337 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 1 -- 172.21.15.19:0/2916523069 >> 172.21.15.19:0/2916523069 conn(0x7f8ba00b05d0 msgr2=0x7f8ba00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:21.337 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 1 -- 172.21.15.19:0/2916523069 shutdown_connections 2023-10-06T13:53:21.337 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 1 -- 172.21.15.19:0/2916523069 wait complete. 2023-10-06T13:53:21.337 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.333+0000 7f8ba588f640 1 librados: shutdown 2023-10-06T13:53:21.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.3 2023-10-06T13:53:21.353 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T13:53:21.379 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T13:53:21.495 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.493+0000 7fc40eafc640 10 monclient: get_monmap_and_config 2023-10-06T13:53:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.493+0000 7fc40eafc640 10 monclient: build_initial_monmap 2023-10-06T13:53:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.493+0000 7fc40eafc640 10 monclient: monmap: 2023-10-06T13:53:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:21.494974+0000 2023-10-06T13:53:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:21.494974+0000 2023-10-06T13:53:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:21.497 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.494+0000 7f7334ecf640 10 monclient: get_monmap_and_config 2023-10-06T13:53:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.494+0000 7f7334ecf640 10 monclient: build_initial_monmap 2023-10-06T13:53:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.494+0000 7fc40eafc640 1 Processor -- start 2023-10-06T13:53:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.494+0000 7f7334ecf640 10 monclient: monmap: 2023-10-06T13:53:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:21.495743+0000 2023-10-06T13:53:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:21.495743+0000 2023-10-06T13:53:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:21.499 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:21.499 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:21.499 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:21.499 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.494+0000 7fc40eafc640 1 -- start start 2023-10-06T13:53:21.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.494+0000 7fc40eafc640 10 monclient: init 2023-10-06T13:53:21.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7f7334ecf640 1 Processor -- start 2023-10-06T13:53:21.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7fc40eafc640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:21.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7fc40eafc640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2023-10-06T13:53:21.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7fc40eafc640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc4080b3990 0x7fc4080af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7f7334ecf640 1 -- start start 2023-10-06T13:53:21.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7f7334ecf640 10 monclient: init 2023-10-06T13:53:21.501 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7fc40eafc640 10 monclient(hunting): picked mon.noname-b con 0x7fc4080b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:21.501 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7fc40eafc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc4080b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:21.501 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7fc40eafc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc4080b00c0 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:21.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7fc40eafc640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4080aebe0 con 0x7fc4080b00c0 2023-10-06T13:53:21.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7fc40eafc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc408141070 con 0x7fc4080b3d60 2023-10-06T13:53:21.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7fc40eafc640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7fc40eafc640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc408140dd0 con 0x7fc4080b3990 2023-10-06T13:53:21.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7fc40eafc640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:21.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.495+0000 7fc40eafc640 10 monclient(hunting): authenticate will time out at 1329.681831s 2023-10-06T13:53:21.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7fc40c898640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc4080b3990 0x7fc4080af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7fc40d099640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc4080b00c0 0x7fc40814b600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57956/0 (socket says 172.21.15.19:57956) 2023-10-06T13:53:21.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7fc40d099640 1 -- 172.21.15.19:0/2384190591 learned_addr learned my addr 172.21.15.19:0/2384190591 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7f7334ecf640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7f7334ecf640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2023-10-06T13:53:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7f7334ecf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f73300b3990 0x7f73300af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7f7334ecf640 10 monclient(hunting): picked mon.noname-a con 0x7f73300b3990 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7f7334ecf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f73300b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7f7334ecf640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f73300b00c0 0x7f73300b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7f7334ecf640 10 monclient(hunting): picked mon.noname-b con 0x7f73300b00c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7f7334ecf640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73300aebe0 con 0x7f73300b3d60 2023-10-06T13:53:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7f7334ecf640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7f7334ecf640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73300afd50 con 0x7f73300b3990 2023-10-06T13:53:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7f7334ecf640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7f7334ecf640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7330140dd0 con 0x7f73300b00c0 2023-10-06T13:53:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7f7334ecf640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7f7334ecf640 10 monclient(hunting): authenticate will time out at 1329.682622s 2023-10-06T13:53:21.505 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7fc40c898640 10 monclient(hunting): get_auth_request con 0x7fc4080b3990 auth_method 0 2023-10-06T13:53:21.505 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7fc40c898640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.505 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7fc40c898640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.505 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.496+0000 7fc40c898640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.505 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.497+0000 7fc40c898640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.505 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.497+0000 7fc40c898640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.505 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.497+0000 7fc40c898640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:21.506 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.497+0000 7f732dd9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f73300b3d60 0x7f7330141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:38390/0 (socket says 172.21.15.19:38390) 2023-10-06T13:53:21.506 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.497+0000 7f732ed9d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f73300b00c0 0x7f73300b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.506 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.497+0000 7f732dd9b640 1 -- 172.21.15.19:0/1635995863 learned_addr learned my addr 172.21.15.19:0/1635995863 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:21.506 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7f732e59c640 1 --2- 172.21.15.19:0/1635995863 >> v2:172.21.15.19:3300/0 conn(0x7f73300b3990 0x7f73300af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.506 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7fc40c898640 10 monclient(hunting): handle_auth_done global_id 4271 payload 274 2023-10-06T13:53:21.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7fc40c898640 1 -- 172.21.15.19:0/2384190591 >> v1:172.21.15.83:6789/0 conn(0x7fc4080b3d60 legacy=0x7fc408141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7fc40c898640 1 -- 172.21.15.19:0/2384190591 >> v1:172.21.15.19:6789/0 conn(0x7fc4080b00c0 legacy=0x7fc40814b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7fc40c898640 10 monclient: _finish_hunting 0 2023-10-06T13:53:21.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7fc40c898640 1 monclient: found mon.noname-b 2023-10-06T13:53:21.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7fc40c898640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:21.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7fc40c898640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T13:53:21.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7fc40c898640 1 -- 172.21.15.19:0/2384190591 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc408141c10 con 0x7fc4080b3990 2023-10-06T13:53:21.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7fc4077fe640 1 -- 172.21.15.19:0/2384190591 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220501168 0 0) 0x7fc4080aebe0 con 0x7fc4080b00c0 2023-10-06T13:53:21.507 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7fc4077fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2023-10-06T13:53:21.508 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7f732e59c640 10 monclient(hunting): get_auth_request con 0x7f73300b3990 auth_method 0 2023-10-06T13:53:21.508 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7f732e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.508 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7f732e59c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.508 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.499+0000 7f732e59c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.508 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.500+0000 7f732ed9d640 10 monclient(hunting): get_auth_request con 0x7f73300b00c0 auth_method 0 2023-10-06T13:53:21.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.500+0000 7f732ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.500+0000 7f732ed9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.500+0000 7f732ed9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.500+0000 7f732e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.500+0000 7f732e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.500+0000 7f732e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:21.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.501+0000 7f732ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.501+0000 7f732ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.501+0000 7f732ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:21.509 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.502+0000 7fc40c898640 1 --2- 172.21.15.19:0/2384190591 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc4080b3990 0x7fc4080af810 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80097d0 tx=0x7fc3f802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6adcf758f673c096 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:21.510 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.503+0000 7f732e59c640 10 monclient(hunting): handle_auth_done global_id 4329 payload 274 2023-10-06T13:53:21.510 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.503+0000 7f732e59c640 1 -- 172.21.15.19:0/1635995863 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f73300b00c0 msgr2=0x7f73300b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:21.510 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.503+0000 7f732e59c640 1 --2- 172.21.15.19:0/1635995863 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f73300b00c0 0x7f73300b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.510 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.503+0000 7f732e59c640 1 -- 172.21.15.19:0/1635995863 >> v1:172.21.15.83:6789/0 conn(0x7f73300b3d60 legacy=0x7f7330141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.503+0000 7f732e59c640 10 monclient: _finish_hunting 0 2023-10-06T13:53:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.503+0000 7f732e59c640 1 monclient: found mon.noname-a 2023-10-06T13:53:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.503+0000 7f732e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.503+0000 7f732e59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.503+0000 7f732e59c640 1 -- 172.21.15.19:0/1635995863 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7330141070 con 0x7f73300b3990 2023-10-06T13:53:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.505+0000 7f732e59c640 1 --2- 172.21.15.19:0/1635995863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73300b3990 0x7f73300af810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f73200097a0 tx=0x7f732002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30dcff42b6818624 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:21.512 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc4077fe640 1 -- 172.21.15.19:0/2384190591 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8031800 con 0x7fc4080b3990 2023-10-06T13:53:21.512 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc4077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.512 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc4077fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:21.512 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc4077fe640 10 monclient: dump: 2023-10-06T13:53:21.512 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.512 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.512 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.512 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.513 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.513 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.513 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.513 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.513 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.513 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.513 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc4077fe640 10 monclient: _finish_auth 0 2023-10-06T13:53:21.513 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc4077fe640 10 monclient: _check_auth_tickets 2023-10-06T13:53:21.513 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc4077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:21.513 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc4077fe640 1 -- 172.21.15.19:0/2384190591 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f8031960 con 0x7fc4080b3990 2023-10-06T13:53:21.513 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc4077fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:21.513 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc4077fe640 1 -- 172.21.15.19:0/2384190591 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8031cf0 con 0x7fc4080b3990 2023-10-06T13:53:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc40eafc640 5 monclient: authenticate success, global_id 4271 2023-10-06T13:53:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc40eafc640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc4077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc40eafc640 4 set_mon_vals no callback set 2023-10-06T13:53:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc4077fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.510+0000 7fc4077fe640 10 monclient: dump: 2023-10-06T13:53:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.515 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.511+0000 7fc40eafc640 10 monclient: shutdown 2023-10-06T13:53:21.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.511+0000 7fc40eafc640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:21.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.511+0000 7fc40eafc640 1 -- 172.21.15.19:0/2384190591 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc4080b3990 msgr2=0x7fc4080af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.511+0000 7fc40eafc640 1 --2- 172.21.15.19:0/2384190591 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc4080b3990 0x7fc4080af810 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80097d0 tx=0x7fc3f802b8a0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.511+0000 7fc40eafc640 1 -- 172.21.15.19:0/2384190591 shutdown_connections 2023-10-06T13:53:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.511+0000 7fc40eafc640 1 --2- 172.21.15.19:0/2384190591 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc4080b3990 0x7fc4080af810 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.511+0000 7fc40eafc640 1 -- 172.21.15.19:0/2384190591 >> 172.21.15.19:0/2384190591 conn(0x7fc4080b6e00 msgr2=0x7fc4080b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.511+0000 7fc40eafc640 1 -- 172.21.15.19:0/2384190591 shutdown_connections 2023-10-06T13:53:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.511+0000 7fc40eafc640 1 -- 172.21.15.19:0/2384190591 wait complete. 2023-10-06T13:53:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7fc40eafc640 10 monclient: build_initial_monmap 2023-10-06T13:53:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7fc40eafc640 10 monclient: monmap: 2023-10-06T13:53:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:21.518 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7fc40eafc640 1 librados: starting msgr at 2023-10-06T13:53:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7fc40eafc640 1 librados: starting objecter 2023-10-06T13:53:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7fc40eafc640 1 Processor -- start 2023-10-06T13:53:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7fc40eafc640 1 -- start start 2023-10-06T13:53:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7fc40eafc640 1 librados: setting wanted keys 2023-10-06T13:53:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7fc40eafc640 1 librados: calling monclient init 2023-10-06T13:53:21.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7fc40eafc640 10 monclient: init 2023-10-06T13:53:21.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f732d59a640 1 -- 172.21.15.19:0/1635995863 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732002e070 con 0x7f73300b3990 2023-10-06T13:53:21.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f732d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f732d59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:21.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f732d59a640 10 monclient: dump: 2023-10-06T13:53:21.519 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.520 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.520 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.520 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.520 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.520 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.520 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.520 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.520 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.520 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f732d59a640 10 monclient: _finish_auth 0 2023-10-06T13:53:21.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f732d59a640 10 monclient: _check_auth_tickets 2023-10-06T13:53:21.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f732d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:21.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f732d59a640 1 -- 172.21.15.19:0/1635995863 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73200315a0 con 0x7f73300b3990 2023-10-06T13:53:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f732d59a640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f732d59a640 1 -- 172.21.15.19:0/1635995863 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320031930 con 0x7f73300b3990 2023-10-06T13:53:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f7334ecf640 5 monclient: authenticate success, global_id 4329 2023-10-06T13:53:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f7334ecf640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f7334ecf640 4 set_mon_vals no callback set 2023-10-06T13:53:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f732d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f732d59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.512+0000 7f732d59a640 10 monclient: dump: 2023-10-06T13:53:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.522 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.522 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.522 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.522 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.522 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.522 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T13:53:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4080b00c0 0x7fc4080abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 10 monclient(hunting): picked mon.noname-b con 0x7fc4080b00c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc4080b3990 0x7fc4080ac6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 10 monclient(hunting): picked mon.noname-c con 0x7fc4080b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4080b3d60 0x7fc4081d6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 10 monclient(hunting): picked mon.noname-a con 0x7fc4080b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4080ace90 con 0x7fc4080b3d60 2023-10-06T13:53:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4080a94a0 con 0x7fc4080b00c0 2023-10-06T13:53:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4080acc50 con 0x7fc4080b3990 2023-10-06T13:53:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40eafc640 10 monclient(hunting): authenticate will time out at 1329.699710s 2023-10-06T13:53:21.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7f7334ecf640 10 monclient: shutdown 2023-10-06T13:53:21.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7f7334ecf640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:21.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7f7334ecf640 1 -- 172.21.15.19:0/1635995863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73300b3990 msgr2=0x7f73300af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7f7334ecf640 1 --2- 172.21.15.19:0/1635995863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73300b3990 0x7f73300af810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f73200097a0 tx=0x7f732002b840 comp rx=0 tx=0).stop 2023-10-06T13:53:21.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7f7334ecf640 1 -- 172.21.15.19:0/1635995863 shutdown_connections 2023-10-06T13:53:21.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7f7334ecf640 1 --2- 172.21.15.19:0/1635995863 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f73300b00c0 0x7f73300b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7f7334ecf640 1 --2- 172.21.15.19:0/1635995863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73300b3990 0x7f73300af810 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7f7334ecf640 1 -- 172.21.15.19:0/1635995863 >> 172.21.15.19:0/1635995863 conn(0x7f73300b6e00 msgr2=0x7f73300b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:21.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7f7334ecf640 1 -- 172.21.15.19:0/1635995863 shutdown_connections 2023-10-06T13:53:21.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7f7334ecf640 1 -- 172.21.15.19:0/1635995863 wait complete. 2023-10-06T13:53:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.513+0000 7fc40c898640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4080b00c0 0x7fc4080abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc407fff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc4080b3990 0x7fc4080ac6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc40c898640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4080b00c0 0x7fc4080abb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39510/0 (socket says 172.21.15.19:39510) 2023-10-06T13:53:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc40c898640 1 -- 172.21.15.19:0/2428521550 learned_addr learned my addr 172.21.15.19:0/2428521550 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc40c898640 10 monclient(hunting): get_auth_request con 0x7fc4080b00c0 auth_method 0 2023-10-06T13:53:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc40c898640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc40c898640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc40c898640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc407fff640 10 monclient(hunting): get_auth_request con 0x7fc4080b3990 auth_method 0 2023-10-06T13:53:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc407fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc407fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc407fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7f7334ecf640 10 monclient: build_initial_monmap 2023-10-06T13:53:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc40d099640 1 --2- 172.21.15.19:0/2428521550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4080b3d60 0x7fc4081d6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc40c898640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc40c898640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7f7334ecf640 10 monclient: monmap: 2023-10-06T13:53:21.529 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:21.529 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:21.529 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:21.529 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:21.529 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:21.529 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.529 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:21.529 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:21.529 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:21.530 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc40c898640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc407fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc407fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.514+0000 7fc407fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:21.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 1 librados: starting msgr at 2023-10-06T13:53:21.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 1 librados: starting objecter 2023-10-06T13:53:21.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 1 Processor -- start 2023-10-06T13:53:21.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 1 -- start start 2023-10-06T13:53:21.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 1 librados: setting wanted keys 2023-10-06T13:53:21.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 1 librados: calling monclient init 2023-10-06T13:53:21.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 10 monclient: init 2023-10-06T13:53:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7fc40d099640 10 monclient(hunting): get_auth_request con 0x7fc4080b3d60 auth_method 0 2023-10-06T13:53:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7fc40d099640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7fc40d099640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7fc40d099640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:21.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T13:53:21.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73300b00c0 0x7f73301427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 10 monclient(hunting): picked mon.noname-b con 0x7f73300b00c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:21.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73300b3990 0x7f7330139060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 10 monclient(hunting): picked mon.noname-a con 0x7f73300b3990 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f73300b3d60 0x7f7330135f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 10 monclient(hunting): picked mon.noname-c con 0x7f73300b3d60 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7330142da0 con 0x7f73300b3990 2023-10-06T13:53:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.515+0000 7f7334ecf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73301375b0 con 0x7f73300b00c0 2023-10-06T13:53:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.516+0000 7f7334ecf640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.516+0000 7f7334ecf640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7330137890 con 0x7f73300b3d60 2023-10-06T13:53:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.516+0000 7f7334ecf640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.516+0000 7f7334ecf640 10 monclient(hunting): authenticate will time out at 1329.702371s 2023-10-06T13:53:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.516+0000 7fc40d099640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.516+0000 7fc40d099640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.516+0000 7fc40d099640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.517+0000 7fc407fff640 10 monclient(hunting): handle_auth_done global_id 4277 payload 824 2023-10-06T13:53:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.517+0000 7fc407fff640 1 -- 172.21.15.19:0/2428521550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4080b00c0 msgr2=0x7fc4080abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.517+0000 7fc407fff640 1 --2- 172.21.15.19:0/2428521550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4080b00c0 0x7fc4080abb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.517+0000 7fc407fff640 1 -- 172.21.15.19:0/2428521550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4080b3d60 msgr2=0x7fc4081d6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.517+0000 7fc407fff640 1 --2- 172.21.15.19:0/2428521550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4080b3d60 0x7fc4081d6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.517+0000 7fc407fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:21.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.517+0000 7fc407fff640 1 monclient: found mon.noname-c 2023-10-06T13:53:21.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.517+0000 7fc407fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:21.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.517+0000 7fc407fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T13:53:21.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.517+0000 7fc407fff640 1 -- 172.21.15.19:0/2428521550 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f8009400 con 0x7fc4080b3990 2023-10-06T13:53:21.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.519+0000 7f732ed9d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f73300b3d60 0x7f7330135f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.520+0000 7f732e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73300b00c0 0x7f73301427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.520+0000 7f732e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73300b00c0 0x7f73301427d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39518/0 (socket says 172.21.15.19:39518) 2023-10-06T13:53:21.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.520+0000 7fc407fff640 1 --2- 172.21.15.19:0/2428521550 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc4080b3990 0x7fc4080ac6e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00c390 tx=0x7fc3fc00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:21.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.520+0000 7fc4057fa640 10 client.?.objecter ms_handle_connect 0x7fc4080b3990 2023-10-06T13:53:21.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.520+0000 7fc4057fa640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:21.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.520+0000 7f732e59c640 1 -- 172.21.15.19:0/4091043265 learned_addr learned my addr 172.21.15.19:0/4091043265 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:21.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.520+0000 7f732e59c640 10 monclient(hunting): get_auth_request con 0x7f73300b00c0 auth_method 0 2023-10-06T13:53:21.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.520+0000 7f732e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.520+0000 7f732e59c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.520+0000 7f732e59c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.520+0000 7f732e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.520+0000 7f732e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.539 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.520+0000 7f732e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:21.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.521+0000 7f732ed9d640 10 monclient(hunting): get_auth_request con 0x7f73300b3d60 auth_method 0 2023-10-06T13:53:21.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.521+0000 7f732ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.521+0000 7f732ed9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.521+0000 7f732ed9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.521+0000 7f732ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.521+0000 7f732ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.521+0000 7f732ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:21.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.522+0000 7f732dd9b640 1 --2- 172.21.15.19:0/4091043265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73300b3990 0x7f7330139060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.522+0000 7f732dd9b640 10 monclient(hunting): get_auth_request con 0x7f73300b3990 auth_method 0 2023-10-06T13:53:21.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.522+0000 7f732dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.522+0000 7f732dd9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.540 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.522+0000 7f732dd9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.522+0000 7f732dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.522+0000 7f732dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.523+0000 7f732dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:21.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.523+0000 7f732e59c640 10 monclient(hunting): handle_auth_done global_id 4360 payload 824 2023-10-06T13:53:21.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.523+0000 7f732e59c640 1 -- 172.21.15.19:0/4091043265 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f73300b3d60 msgr2=0x7f7330135f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.523+0000 7f732e59c640 1 --2- 172.21.15.19:0/4091043265 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f73300b3d60 0x7f7330135f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.523+0000 7f732e59c640 1 -- 172.21.15.19:0/4091043265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73300b3990 msgr2=0x7f7330139060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.523+0000 7f732e59c640 1 --2- 172.21.15.19:0/4091043265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73300b3990 0x7f7330139060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.523+0000 7f732e59c640 10 monclient: _finish_hunting 0 2023-10-06T13:53:21.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.523+0000 7f732e59c640 1 monclient: found mon.noname-b 2023-10-06T13:53:21.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.523+0000 7f732e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:21.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.523+0000 7f732e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:21.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.523+0000 7f732e59c640 1 -- 172.21.15.19:0/4091043265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7320009400 con 0x7f73300b00c0 2023-10-06T13:53:21.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.526+0000 7f732e59c640 1 --2- 172.21.15.19:0/4091043265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73300b00c0 0x7f73301427d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7320034040 tx=0x7f73200046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:21.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.526+0000 7f731effd640 10 client.?.objecter ms_handle_connect 0x7f73300b00c0 2023-10-06T13:53:21.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.526+0000 7f731effd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:21.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc4057fa640 1 -- 172.21.15.19:0/2428521550 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0052d0 con 0x7fc4080b3990 2023-10-06T13:53:21.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc4057fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc4057fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:21.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc4057fa640 10 monclient: dump: 2023-10-06T13:53:21.543 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.543 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.543 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.543 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.544 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.544 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.544 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.544 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.544 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.544 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc4057fa640 10 monclient: _finish_auth 0 2023-10-06T13:53:21.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc4057fa640 10 monclient: _check_auth_tickets 2023-10-06T13:53:21.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc4057fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:21.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc4057fa640 1 -- 172.21.15.19:0/2428521550 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc005430 con 0x7fc4080b3990 2023-10-06T13:53:21.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc40eafc640 5 monclient: authenticate success, global_id 4277 2023-10-06T13:53:21.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc40eafc640 10 monclient: _renew_subs 2023-10-06T13:53:21.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc40eafc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:21.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc40eafc640 1 -- 172.21.15.19:0/2428521550 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4080aae00 con 0x7fc4080b3990 2023-10-06T13:53:21.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc4057fa640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:21.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc4057fa640 1 -- 172.21.15.19:0/2428521550 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc003830 con 0x7fc4080b3990 2023-10-06T13:53:21.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc4057fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc4057fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:21.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc4057fa640 10 monclient: dump: 2023-10-06T13:53:21.545 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.545 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.545 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.545 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.545 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.545 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.546 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.546 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.546 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.546 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc406ffd640 4 set_mon_vals no callback set 2023-10-06T13:53:21.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc40eafc640 10 client.4277.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:21.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc40eafc640 10 monclient: _renew_subs 2023-10-06T13:53:21.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc40eafc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:21.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc40eafc640 1 -- 172.21.15.19:0/2428521550 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4081d6740 con 0x7fc4080b3990 2023-10-06T13:53:21.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.529+0000 7fc40eafc640 1 librados: init done 2023-10-06T13:53:21.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.531+0000 7fc40eafc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:21.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.531+0000 7fc40eafc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:21.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.531+0000 7fc40eafc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:21.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.531+0000 7fc40eafc640 1 -- 172.21.15.19:0/2428521550 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc408140bf0 con 0x7fc4080b3990 2023-10-06T13:53:21.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.550+0000 7fc4057fa640 1 -- 172.21.15.19:0/2428521550 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc3fc003990 con 0x7fc4080b3990 2023-10-06T13:53:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.550+0000 7fc4057fa640 1 --2- 172.21.15.19:0/2428521550 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc3f406c210 0x7fc3f406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.551+0000 7fc40c898640 1 --2- 172.21.15.19:0/2428521550 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc3f406c210 0x7fc3f406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.551+0000 7fc40c898640 10 monclient: get_auth_request con 0x7fc3f406c210 auth_method 0 2023-10-06T13:53:21.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.551+0000 7fc40c898640 1 --2- 172.21.15.19:0/2428521550 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc3f406c210 0x7fc3f406e6c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc3f802b8e0 tx=0x7fc3f803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:21.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.551+0000 7fc4057fa640 10 client.4277.objecter ms_handle_connect 0x7fc3f406c210 2023-10-06T13:53:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.552+0000 7fc4057fa640 1 -- 172.21.15.19:0/2428521550 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc3fc057670 con 0x7fc4080b3990 2023-10-06T13:53:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.552+0000 7fc4057fa640 10 client.4277.objecter ms_dispatch 0x7fc408000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.552+0000 7fc4057fa640 3 client.4277.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.552+0000 7fc4057fa640 3 client.4277.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.553+0000 7fc4057fa640 20 client.4277.objecter dump_active .. 0 homeless 2023-10-06T13:53:21.590 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T13:53:21.617 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T13:53:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.646+0000 7f731effd640 1 -- 172.21.15.19:0/4091043265 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320032840 con 0x7f73300b00c0 2023-10-06T13:53:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f731effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f731effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f731effd640 10 monclient: dump: 2023-10-06T13:53:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.650 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.650 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.650 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.650 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.650 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.650 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f731effd640 10 monclient: _finish_auth 0 2023-10-06T13:53:21.650 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f731effd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:21.650 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f731effd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:21.650 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f731effd640 1 -- 172.21.15.19:0/4091043265 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73200329c0 con 0x7f73300b00c0 2023-10-06T13:53:21.650 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f7334ecf640 5 monclient: authenticate success, global_id 4360 2023-10-06T13:53:21.650 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f731effd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:21.650 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f7334ecf640 10 monclient: _renew_subs 2023-10-06T13:53:21.651 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f731effd640 1 -- 172.21.15.19:0/4091043265 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320032d70 con 0x7f73300b00c0 2023-10-06T13:53:21.651 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f7334ecf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:21.651 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f7334ecf640 1 -- 172.21.15.19:0/4091043265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73301420d0 con 0x7f73300b00c0 2023-10-06T13:53:21.651 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f731effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.651 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f731effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:21.651 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f731effd640 10 monclient: dump: 2023-10-06T13:53:21.651 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.651 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.651 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.651 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.651 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.651 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.652 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.652 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.652 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.652 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.652 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f732cd99640 4 set_mon_vals no callback set 2023-10-06T13:53:21.652 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f7334ecf640 10 client.4360.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:21.652 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f7334ecf640 10 monclient: _renew_subs 2023-10-06T13:53:21.652 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f7334ecf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:21.652 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f7334ecf640 1 -- 172.21.15.19:0/4091043265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73301364a0 con 0x7f73300b00c0 2023-10-06T13:53:21.652 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.647+0000 7f7334ecf640 1 librados: init done 2023-10-06T13:53:21.653 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.648+0000 7f7334ecf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:21.653 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.648+0000 7f7334ecf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:21.653 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.648+0000 7f7334ecf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:21.653 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.648+0000 7f7334ecf640 1 -- 172.21.15.19:0/4091043265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73300678e0 con 0x7f73300b00c0 2023-10-06T13:53:21.700 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.698+0000 7f491589d640 10 monclient: get_monmap_and_config 2023-10-06T13:53:21.700 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.698+0000 7f491589d640 10 monclient: build_initial_monmap 2023-10-06T13:53:21.700 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.698+0000 7f491589d640 10 monclient: monmap: 2023-10-06T13:53:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:21.699754+0000 2023-10-06T13:53:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:21.699754+0000 2023-10-06T13:53:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:21.701 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.699+0000 7f491589d640 1 Processor -- start 2023-10-06T13:53:21.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.699+0000 7f491589d640 1 -- start start 2023-10-06T13:53:21.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.699+0000 7f491589d640 10 monclient: init 2023-10-06T13:53:21.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.699+0000 7f491589d640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:21.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.700+0000 7f491589d640 10 monclient: _add_conns ranks=[1,0,4,2,3] 2023-10-06T13:53:21.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.700+0000 7f491589d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f49100b3990 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:21.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.700+0000 7f491589d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f49100b3d60 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:21.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.700+0000 7f491589d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f49100b07d0 0x7f49100af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.700+0000 7f491589d640 10 monclient(hunting): picked mon.noname-c con 0x7f49100b07d0 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:21.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.700+0000 7f491589d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49100aebe0 con 0x7f49100b3d60 2023-10-06T13:53:21.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.700+0000 7f491589d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4910141070 con 0x7f49100b3990 2023-10-06T13:53:21.704 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.700+0000 7f491589d640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.704 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.700+0000 7f491589d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4910140dd0 con 0x7f49100b07d0 2023-10-06T13:53:21.704 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.700+0000 7f491589d640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:21.704 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.700+0000 7f491589d640 10 monclient(hunting): authenticate will time out at 1329.886479s 2023-10-06T13:53:21.704 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.700+0000 7f490e7fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f49100b3d60 0x7f49100b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57972/0 (socket says 172.21.15.19:57972) 2023-10-06T13:53:21.704 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.700+0000 7f490e7fc640 1 -- 172.21.15.19:0/3391241448 learned_addr learned my addr 172.21.15.19:0/3391241448 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:21.704 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.703+0000 7f490dffb640 1 -- 172.21.15.19:0/3391241448 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427971332 0 0) 0x7f49100aebe0 con 0x7f49100b3d60 2023-10-06T13:53:21.705 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.703+0000 7f490dffb640 10 monclient(hunting): _negotiate 2023-10-06T13:53:21.705 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.703+0000 7f490dffb640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.705 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.703+0000 7f490dffb640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.705 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.703+0000 7f490dffb640 10 monclient(hunting): my global_id is 4338 2023-10-06T13:53:21.705 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.703+0000 7f490dffb640 1 -- 172.21.15.19:0/3391241448 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48f8003520 con 0x7f49100b3d60 2023-10-06T13:53:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 1 -- 172.21.15.19:0/3391241448 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3345387193 0 0) 0x7f48f8003520 con 0x7f49100b3d60 2023-10-06T13:53:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 1 -- 172.21.15.19:0/3391241448 >> v2:172.21.15.83:3300/0 conn(0x7f49100b07d0 msgr2=0x7f49100af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 1 --2- 172.21.15.19:0/3391241448 >> v2:172.21.15.83:3300/0 conn(0x7f49100b07d0 0x7f49100af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 1 -- 172.21.15.19:0/3391241448 >> v1:172.21.15.83:6789/0 conn(0x7f49100b3990 legacy=0x7f4910141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 10 monclient: _finish_hunting 0 2023-10-06T13:53:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 1 monclient: found mon.noname-a-legacy 2023-10-06T13:53:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.19:6789/0 2023-10-06T13:53:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 1 -- 172.21.15.19:0/3391241448 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4910141c10 con 0x7f49100b3d60 2023-10-06T13:53:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 10 monclient: _finish_auth 0 2023-10-06T13:53:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 10 monclient: _check_auth_tickets 2023-10-06T13:53:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 1 -- 172.21.15.19:0/3391241448 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251531364 0 0) 0x7f4904001670 con 0x7f49100b3d60 2023-10-06T13:53:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f491589d640 5 monclient: authenticate success, global_id 4338 2023-10-06T13:53:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f491589d640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2023-10-06T13:53:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 10 monclient: dump: 2023-10-06T13:53:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.711 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.711 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.711 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.711 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.711 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.711 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] but i'm connected to v1:172.21.15.19:6789/0, reconnecting 2023-10-06T13:53:21.711 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:21.711 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 1 -- 172.21.15.19:0/3391241448 >> v1:172.21.15.19:6789/0 conn(0x7f49100b3d60 legacy=0x7f49100b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.711 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:53:21.711 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 1 --2- 172.21.15.19:0/3391241448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48f8003970 0x7f48f8003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.711 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.706+0000 7f490dffb640 10 monclient(hunting): picked mon.a con 0x7f48f8003970 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:21.711 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.707+0000 7f490dffb640 1 --2- 172.21.15.19:0/3391241448 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f48f80047c0 0x7f48f8004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.707+0000 7f490dffb640 10 monclient(hunting): picked mon.c con 0x7f48f80047c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.707+0000 7f490dffb640 1 --2- 172.21.15.19:0/3391241448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48f80052c0 0x7f48f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.707+0000 7f490dffb640 10 monclient(hunting): picked mon.b con 0x7f48f80052c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.707+0000 7f490dffb640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.707+0000 7f490dffb640 1 -- 172.21.15.19:0/3391241448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4910140dd0 con 0x7f48f8003970 2023-10-06T13:53:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.707+0000 7f490dffb640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.707+0000 7f490dffb640 1 -- 172.21.15.19:0/3391241448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f800c1f0 con 0x7f48f80052c0 2023-10-06T13:53:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.707+0000 7f490dffb640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.707+0000 7f490dffb640 1 -- 172.21.15.19:0/3391241448 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f800c3b0 con 0x7f48f80047c0 2023-10-06T13:53:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.707+0000 7f490dffb640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.707+0000 7f490dffb640 10 monclient(hunting): _finish_auth 0 2023-10-06T13:53:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.707+0000 7f491589d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:21.713 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.709+0000 7f490effd640 1 --2- 172.21.15.19:0/3391241448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48f8003970 0x7f48f8003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.713 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.710+0000 7f490effd640 10 monclient(hunting): get_auth_request con 0x7f48f8003970 auth_method 0 2023-10-06T13:53:21.713 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.710+0000 7f490effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.713 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.710+0000 7f490effd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.713 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.710+0000 7f490effd640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:53:21.713 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.710+0000 7f490effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.713 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.710+0000 7f490effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.714 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.710+0000 7f490effd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T13:53:21.714 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.712+0000 7f490effd640 10 monclient(hunting): handle_auth_done global_id 4338 payload 293 2023-10-06T13:53:21.714 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.712+0000 7f490effd640 1 -- 172.21.15.19:0/3391241448 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f48f80047c0 msgr2=0x7f48f8004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:21.714 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.712+0000 7f490effd640 1 --2- 172.21.15.19:0/3391241448 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f48f80047c0 0x7f48f8004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.712+0000 7f490effd640 1 -- 172.21.15.19:0/3391241448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48f80052c0 msgr2=0x7f48f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.712+0000 7f490effd640 1 --2- 172.21.15.19:0/3391241448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48f80052c0 0x7f48f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.712+0000 7f490effd640 10 monclient: _finish_hunting 0 2023-10-06T13:53:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.712+0000 7f490effd640 1 monclient: found mon.a 2023-10-06T13:53:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.712+0000 7f490effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.712+0000 7f490effd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.712+0000 7f490effd640 1 -- 172.21.15.19:0/3391241448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f8003520 con 0x7f48f8003970 2023-10-06T13:53:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.712+0000 7f490f7fe640 1 -- 172.21.15.19:0/3391241448 reap_dead start 2023-10-06T13:53:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.712+0000 7f490effd640 10 monclient: _finish_auth 0 2023-10-06T13:53:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.712+0000 7f490effd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.712+0000 7f490effd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.714+0000 7f490effd640 1 --2- 172.21.15.19:0/3391241448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48f8003970 0x7f48f8003d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4900001650 tx=0x7f48f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ae421f9e37ed9d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:21.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.718+0000 7f490dffb640 1 -- 172.21.15.19:0/3391241448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900005270 con 0x7f48f8003970 2023-10-06T13:53:21.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.718+0000 7f490dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.718+0000 7f490dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:21.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.718+0000 7f490dffb640 10 monclient: dump: 2023-10-06T13:53:21.720 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.720 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.720 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.721 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.718+0000 7f491589d640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.718+0000 7f490dffb640 1 -- 172.21.15.19:0/3391241448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490002e070 con 0x7f48f8003970 2023-10-06T13:53:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.718+0000 7f490dffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.718+0000 7f490dffb640 1 -- 172.21.15.19:0/3391241448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900003610 con 0x7f48f8003970 2023-10-06T13:53:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.718+0000 7f491589d640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.718+0000 7f490dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.718+0000 7f491589d640 4 set_mon_vals no callback set 2023-10-06T13:53:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.718+0000 7f490dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.718+0000 7f490dffb640 10 monclient: dump: 2023-10-06T13:53:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.722 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.723 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.719+0000 7f491589d640 10 monclient: shutdown 2023-10-06T13:53:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.719+0000 7f491589d640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.719+0000 7f491589d640 1 -- 172.21.15.19:0/3391241448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48f8003970 msgr2=0x7f48f8003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.719+0000 7f491589d640 1 --2- 172.21.15.19:0/3391241448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48f8003970 0x7f48f8003d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4900001650 tx=0x7f48f80057f0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.719+0000 7f491589d640 1 -- 172.21.15.19:0/3391241448 shutdown_connections 2023-10-06T13:53:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.719+0000 7f491589d640 1 --2- 172.21.15.19:0/3391241448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48f8003970 0x7f48f8003d60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.719+0000 7f491589d640 1 -- 172.21.15.19:0/3391241448 >> 172.21.15.19:0/3391241448 conn(0x7f49100b6e00 msgr2=0x7f49100b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.719+0000 7f491589d640 1 -- 172.21.15.19:0/3391241448 shutdown_connections 2023-10-06T13:53:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.720+0000 7f491589d640 1 -- 172.21.15.19:0/3391241448 wait complete. 2023-10-06T13:53:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.720+0000 7f491589d640 10 monclient: build_initial_monmap 2023-10-06T13:53:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.720+0000 7f491589d640 10 monclient: monmap: 2023-10-06T13:53:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:21.726 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.721+0000 7f491589d640 1 librados: starting msgr at 2023-10-06T13:53:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.721+0000 7f491589d640 1 librados: starting objecter 2023-10-06T13:53:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.721+0000 7f491589d640 1 Processor -- start 2023-10-06T13:53:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.721+0000 7f491589d640 1 -- start start 2023-10-06T13:53:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.721+0000 7f491589d640 1 librados: setting wanted keys 2023-10-06T13:53:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.721+0000 7f491589d640 1 librados: calling monclient init 2023-10-06T13:53:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.721+0000 7f491589d640 10 monclient: init 2023-10-06T13:53:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T13:53:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48f8003970 0x7f4910144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 10 monclient(hunting): picked mon.noname-a con 0x7f48f8003970 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f491013a260 0x7f49100acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 10 monclient(hunting): picked mon.noname-b con 0x7f491013a260 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f49101395f0 0x7f4910139a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 10 monclient(hunting): picked mon.noname-c con 0x7f49101395f0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49100a96f0 con 0x7f48f8003970 2023-10-06T13:53:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49100ad260 con 0x7f491013a260 2023-10-06T13:53:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4910139fa0 con 0x7f49101395f0 2023-10-06T13:53:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.722+0000 7f491589d640 10 monclient(hunting): authenticate will time out at 1329.909155s 2023-10-06T13:53:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.723+0000 7f490effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48f8003970 0x7f4910144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.723+0000 7f490effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48f8003970 0x7f4910144800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39010/0 (socket says 172.21.15.19:39010) 2023-10-06T13:53:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.723+0000 7f490effd640 1 -- 172.21.15.19:0/4068037190 learned_addr learned my addr 172.21.15.19:0/4068037190 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:21.729 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.723+0000 7f490effd640 10 monclient(hunting): get_auth_request con 0x7f48f8003970 auth_method 0 2023-10-06T13:53:21.729 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.723+0000 7f490effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.729 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.723+0000 7f490effd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.729 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.723+0000 7f490effd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.724+0000 7f490effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.724+0000 7f490effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.724+0000 7f490effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.727+0000 7f490effd640 10 monclient(hunting): handle_auth_done global_id 4341 payload 824 2023-10-06T13:53:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.727+0000 7f490effd640 1 -- 172.21.15.19:0/4068037190 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f49101395f0 msgr2=0x7f4910139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.727+0000 7f490effd640 1 --2- 172.21.15.19:0/4068037190 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f49101395f0 0x7f4910139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.727+0000 7f490effd640 1 -- 172.21.15.19:0/4068037190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f491013a260 msgr2=0x7f49100acbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.727+0000 7f490effd640 1 --2- 172.21.15.19:0/4068037190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f491013a260 0x7f49100acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.727+0000 7f490effd640 10 monclient: _finish_hunting 0 2023-10-06T13:53:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.727+0000 7f490effd640 1 monclient: found mon.noname-a 2023-10-06T13:53:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.727+0000 7f490effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:21.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.727+0000 7f490effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:21.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.727+0000 7f490effd640 1 -- 172.21.15.19:0/4068037190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49101d9b10 con 0x7f48f8003970 2023-10-06T13:53:21.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.729+0000 7f490effd640 1 --2- 172.21.15.19:0/4068037190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48f8003970 0x7f4910144800 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f490002bb10 tx=0x7f4900000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:21.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.729+0000 7f48f7fff640 10 client.?.objecter ms_handle_connect 0x7f48f8003970 2023-10-06T13:53:21.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.729+0000 7f48f7fff640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f48f7fff640 1 -- 172.21.15.19:0/4068037190 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900007240 con 0x7f48f8003970 2023-10-06T13:53:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f48f7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f48f7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f48f7fff640 10 monclient: dump: 2023-10-06T13:53:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.737 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f48f7fff640 10 monclient: _finish_auth 0 2023-10-06T13:53:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f48f7fff640 10 monclient: _check_auth_tickets 2023-10-06T13:53:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f48f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f48f7fff640 1 -- 172.21.15.19:0/4068037190 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49000073a0 con 0x7f48f8003970 2023-10-06T13:53:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f491589d640 5 monclient: authenticate success, global_id 4341 2023-10-06T13:53:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f491589d640 10 monclient: _renew_subs 2023-10-06T13:53:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f491589d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f491589d640 1 -- 172.21.15.19:0/4068037190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49101d9d00 con 0x7f48f8003970 2023-10-06T13:53:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f48f7fff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f48f7fff640 1 -- 172.21.15.19:0/4068037190 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49000067b0 con 0x7f48f8003970 2023-10-06T13:53:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f48f7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f48f7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f48f7fff640 10 monclient: dump: 2023-10-06T13:53:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.739 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f490d7fa640 4 set_mon_vals no callback set 2023-10-06T13:53:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f491589d640 10 client.4341.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f491589d640 10 monclient: _renew_subs 2023-10-06T13:53:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f491589d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f491589d640 1 -- 172.21.15.19:0/4068037190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49101da210 con 0x7f48f8003970 2023-10-06T13:53:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.734+0000 7f491589d640 1 librados: init done 2023-10-06T13:53:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.735+0000 7f491589d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.735+0000 7f491589d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.735+0000 7f491589d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.735+0000 7f491589d640 1 -- 172.21.15.19:0/4068037190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49100678e0 con 0x7f48f8003970 2023-10-06T13:53:21.744 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.742+0000 7f9d1d246640 10 monclient: get_monmap_and_config 2023-10-06T13:53:21.744 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.742+0000 7f9d1d246640 10 monclient: build_initial_monmap 2023-10-06T13:53:21.744 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.742+0000 7f9d1d246640 10 monclient: monmap: 2023-10-06T13:53:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:21.743827+0000 2023-10-06T13:53:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:21.743827+0000 2023-10-06T13:53:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:21.745 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.743+0000 7f9d1d246640 1 Processor -- start 2023-10-06T13:53:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.743+0000 7f9d1d246640 1 -- start start 2023-10-06T13:53:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.743+0000 7f9d1d246640 10 monclient: init 2023-10-06T13:53:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.743+0000 7f9d1d246640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.743+0000 7f9d1d246640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2023-10-06T13:53:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.743+0000 7f9d1d246640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9d180b3990 0x7f9d180af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.744+0000 7f9d1d246640 10 monclient(hunting): picked mon.noname-a con 0x7f9d180b3990 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.744+0000 7f9d1d246640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9d180b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.744+0000 7f9d1d246640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9d180b00c0 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.744+0000 7f9d1d246640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d180aebe0 con 0x7f9d180b00c0 2023-10-06T13:53:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.744+0000 7f9d1d246640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d18141070 con 0x7f9d180b3d60 2023-10-06T13:53:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.744+0000 7f9d1d246640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.744+0000 7f9d1d246640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d18140dd0 con 0x7f9d180b3990 2023-10-06T13:53:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.744+0000 7f9d1d246640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.744+0000 7f9d1d246640 10 monclient(hunting): authenticate will time out at 1329.930444s 2023-10-06T13:53:21.766 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.764+0000 7f48f7fff640 1 -- 172.21.15.19:0/4068037190 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f4900030030 con 0x7f48f8003970 2023-10-06T13:53:21.766 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.764+0000 7f48f7fff640 1 --2- 172.21.15.19:0/4068037190 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f48fc06c210 0x7f48fc06e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.767 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.765+0000 7f490e7fc640 1 --2- 172.21.15.19:0/4068037190 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f48fc06c210 0x7f48fc06e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.767 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.765+0000 7f490e7fc640 10 monclient: get_auth_request con 0x7f48fc06c210 auth_method 0 2023-10-06T13:53:21.767 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.765+0000 7f490e7fc640 1 --2- 172.21.15.19:0/4068037190 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f48fc06c210 0x7f48fc06e6c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4904000fd0 tx=0x7f4904001590 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:21.767 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.765+0000 7f48f7fff640 10 client.4341.objecter ms_handle_connect 0x7f48fc06c210 2023-10-06T13:53:21.811 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T13:53:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.819+0000 7fc4057fa640 1 -- 172.21.15.19:0/2428521550 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7fc3fc0539b0 con 0x7fc4080b3990 2023-10-06T13:53:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.819+0000 7fc4057fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.819+0000 7fc4057fa640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:21.901 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.900+0000 7f731effd640 1 -- 172.21.15.19:0/4091043265 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f732004e430 con 0x7f73300b00c0 2023-10-06T13:53:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.900+0000 7f731effd640 1 --2- 172.21.15.19:0/4091043265 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f731006c210 0x7f731006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.900+0000 7f732dd9b640 1 --2- 172.21.15.19:0/4091043265 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f731006c210 0x7f731006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.900+0000 7f732dd9b640 10 monclient: get_auth_request con 0x7f731006c210 auth_method 0 2023-10-06T13:53:21.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.901+0000 7f731effd640 10 client.4360.objecter ms_handle_connect 0x7f731006c210 2023-10-06T13:53:21.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.901+0000 7f732dd9b640 1 --2- 172.21.15.19:0/4091043265 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f731006c210 0x7f731006e6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f733013eaa0 tx=0x7f7324009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.903+0000 7f731effd640 1 -- 172.21.15.19:0/4091043265 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f732007a3c0 con 0x7f73300b00c0 2023-10-06T13:53:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.903+0000 7f731effd640 10 client.4360.objecter ms_dispatch 0x7f7330000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.903+0000 7f731effd640 3 client.4360.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.903+0000 7f731effd640 3 client.4360.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.903+0000 7f731effd640 20 client.4360.objecter dump_active .. 0 homeless 2023-10-06T13:53:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.915+0000 7f731effd640 1 -- 172.21.15.19:0/4091043265 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f7320076700 con 0x7f73300b00c0 2023-10-06T13:53:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.915+0000 7f731effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.915+0000 7f731effd640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:21.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.919+0000 7f9d1659c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9d180b3d60 0x7f9d18141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:38402/0 (socket says 172.21.15.19:38402) 2023-10-06T13:53:21.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.919+0000 7f9d1659c640 1 -- 172.21.15.19:0/3002850128 learned_addr learned my addr 172.21.15.19:0/3002850128 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:21.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.920+0000 7fe07a0b6640 10 monclient: get_monmap_and_config 2023-10-06T13:53:21.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.920+0000 7fe07a0b6640 10 monclient: build_initial_monmap 2023-10-06T13:53:21.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.920+0000 7fe07a0b6640 10 monclient: monmap: 2023-10-06T13:53:21.922 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:21.922 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.922 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:21.921538+0000 2023-10-06T13:53:21.922 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:21.921538+0000 2023-10-06T13:53:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:21.923 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.921+0000 7fe07a0b6640 1 Processor -- start 2023-10-06T13:53:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.921+0000 7fe07a0b6640 1 -- start start 2023-10-06T13:53:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.921+0000 7fe07a0b6640 10 monclient: init 2023-10-06T13:53:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.921+0000 7fe07a0b6640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.921+0000 7fe07a0b6640 10 monclient: _add_conns ranks=[3,0,1,4,2] 2023-10-06T13:53:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.921+0000 7fe07a0b6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe0740b3990 0x7fe0740af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.921+0000 7fe07a0b6640 10 monclient(hunting): picked mon.noname-a con 0x7fe0740b3990 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.922+0000 7fe07a0b6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe0740b3d60 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.922+0000 7fe07a0b6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe0740b00c0 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.922+0000 7fe07a0b6640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0740aebe0 con 0x7fe0740b3d60 2023-10-06T13:53:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.922+0000 7fe07a0b6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe074141070 con 0x7fe0740b00c0 2023-10-06T13:53:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.922+0000 7fe07a0b6640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.922+0000 7fe07a0b6640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe074140dd0 con 0x7fe0740b3990 2023-10-06T13:53:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.922+0000 7fe07a0b6640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.922+0000 7fe07a0b6640 10 monclient(hunting): authenticate will time out at 1330.108446s 2023-10-06T13:53:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.923+0000 7f9d15d9b640 1 -- 172.21.15.19:0/3002850128 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646254431 0 0) 0x7f9d18141070 con 0x7f9d180b3d60 2023-10-06T13:53:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.923+0000 7f9d15d9b640 10 monclient(hunting): _negotiate 2023-10-06T13:53:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.923+0000 7f9d15d9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.923+0000 7f9d15d9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.923+0000 7f9d15d9b640 10 monclient(hunting): my global_id is 4363 2023-10-06T13:53:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.923+0000 7f9d15d9b640 1 -- 172.21.15.19:0/3002850128 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d00003520 con 0x7f9d180b3d60 2023-10-06T13:53:21.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.926+0000 7fe073fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe0740b00c0 0x7fe07414b600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:38408/0 (socket says 172.21.15.19:38408) 2023-10-06T13:53:21.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.926+0000 7fe073fff640 1 -- 172.21.15.19:0/3197356778 learned_addr learned my addr 172.21.15.19:0/3197356778 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 -- 172.21.15.19:0/3002850128 <== mon.1 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3215688306 0 0) 0x7f9d00003520 con 0x7f9d180b3d60 2023-10-06T13:53:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 -- 172.21.15.19:0/3002850128 >> v2:172.21.15.19:3300/0 conn(0x7f9d180b3990 msgr2=0x7f9d180af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 --2- 172.21.15.19:0/3002850128 >> v2:172.21.15.19:3300/0 conn(0x7f9d180b3990 0x7f9d180af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 -- 172.21.15.19:0/3002850128 >> v1:172.21.15.19:6789/0 conn(0x7f9d180b00c0 legacy=0x7f9d1814b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient: _finish_hunting 0 2023-10-06T13:53:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 monclient: found mon.noname-c-legacy 2023-10-06T13:53:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.83:6789/0 2023-10-06T13:53:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 -- 172.21.15.19:0/3002850128 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d18141c10 con 0x7f9d180b3d60 2023-10-06T13:53:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient: _finish_auth 0 2023-10-06T13:53:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient: _check_auth_tickets 2023-10-06T13:53:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 -- 172.21.15.19:0/3002850128 <== mon.1 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251531364 0 0) 0x7f9d0c001670 con 0x7f9d180b3d60 2023-10-06T13:53:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2023-10-06T13:53:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient: dump: 2023-10-06T13:53:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.932 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] but i'm connected to v1:172.21.15.83:6789/0, reconnecting 2023-10-06T13:53:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 -- 172.21.15.19:0/3002850128 >> v1:172.21.15.83:6789/0 conn(0x7f9d180b3d60 legacy=0x7f9d18141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient: _add_conns ranks=[2,0,1] 2023-10-06T13:53:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 --2- 172.21.15.19:0/3002850128 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9d00003970 0x7f9d00003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient(hunting): picked mon.c con 0x7f9d00003970 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 --2- 172.21.15.19:0/3002850128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d000047c0 0x7f9d00004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient(hunting): picked mon.a con 0x7f9d000047c0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 --2- 172.21.15.19:0/3002850128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d000052c0 0x7f9d0000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient(hunting): picked mon.b con 0x7f9d000052c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 -- 172.21.15.19:0/3002850128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d18140dd0 con 0x7f9d000047c0 2023-10-06T13:53:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 -- 172.21.15.19:0/3002850128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0000c1f0 con 0x7f9d000052c0 2023-10-06T13:53:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 1 -- 172.21.15.19:0/3002850128 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0000c3b0 con 0x7f9d00003970 2023-10-06T13:53:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.927+0000 7f9d15d9b640 10 monclient(hunting): _finish_auth 0 2023-10-06T13:53:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.929+0000 7f9d16d9d640 1 --2- 172.21.15.19:0/3002850128 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9d00003970 0x7f9d00003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.930+0000 7f9d16d9d640 10 monclient(hunting): get_auth_request con 0x7f9d00003970 auth_method 0 2023-10-06T13:53:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.930+0000 7f9d16d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.930+0000 7f9d16d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.930+0000 7f9d16d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:53:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.930+0000 7f9d1759e640 1 --2- 172.21.15.19:0/3002850128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d000052c0 0x7f9d0000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.930+0000 7f9d16d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.930+0000 7f9d16d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.930+0000 7f9d16d9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T13:53:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.932+0000 7f9d1759e640 10 monclient(hunting): get_auth_request con 0x7f9d000052c0 auth_method 0 2023-10-06T13:53:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.932+0000 7f9d1759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.932+0000 7f9d1759e640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.932+0000 7f9d1759e640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:53:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.933+0000 7f9d1759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.933+0000 7f9d1759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.933+0000 7f9d1759e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T13:53:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d16d9d640 10 monclient(hunting): handle_auth_done global_id 4363 payload 293 2023-10-06T13:53:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d16d9d640 1 -- 172.21.15.19:0/3002850128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d000052c0 msgr2=0x7f9d0000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d16d9d640 1 --2- 172.21.15.19:0/3002850128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d000052c0 0x7f9d0000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d16d9d640 1 -- 172.21.15.19:0/3002850128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d000047c0 msgr2=0x7f9d00004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d16d9d640 1 --2- 172.21.15.19:0/3002850128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d000047c0 0x7f9d00004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d1759e640 1 -- 172.21.15.19:0/3002850128 reap_dead start 2023-10-06T13:53:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d16d9d640 10 monclient: _finish_hunting 0 2023-10-06T13:53:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d16d9d640 1 monclient: found mon.c 2023-10-06T13:53:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d16d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d16d9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d16d9d640 1 -- 172.21.15.19:0/3002850128 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d00003520 con 0x7f9d00003970 2023-10-06T13:53:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d16d9d640 10 monclient: _finish_auth 0 2023-10-06T13:53:21.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d16d9d640 10 monclient: _check_auth_tickets 2023-10-06T13:53:21.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d16d9d640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:21.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d1d246640 5 monclient: authenticate success, global_id 4363 2023-10-06T13:53:21.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.934+0000 7f9d1d246640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:21.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.937+0000 7fe0727fc640 1 -- 172.21.15.19:0/3197356778 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125000000 0 0) 0x7fe074141070 con 0x7fe0740b00c0 2023-10-06T13:53:21.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.937+0000 7fe0727fc640 10 monclient(hunting): _negotiate 2023-10-06T13:53:21.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.937+0000 7fe0727fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.937+0000 7fe0727fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.937+0000 7fe0727fc640 10 monclient(hunting): my global_id is 4366 2023-10-06T13:53:21.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.937+0000 7fe0727fc640 1 -- 172.21.15.19:0/3197356778 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe05c003520 con 0x7fe0740b00c0 2023-10-06T13:53:21.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.937+0000 7f9d16d9d640 1 --2- 172.21.15.19:0/3002850128 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9d00003970 0x7f9d00003d60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9d1804f900 tx=0x7f9d000057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b8c5ab328fe2b6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:21.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 -- 172.21.15.19:0/3197356778 <== mon.1 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 104538183 0 0) 0x7fe05c003520 con 0x7fe0740b00c0 2023-10-06T13:53:21.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 -- 172.21.15.19:0/3197356778 >> v2:172.21.15.19:3300/0 conn(0x7fe0740b3990 msgr2=0x7fe0740af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:21.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 --2- 172.21.15.19:0/3197356778 >> v2:172.21.15.19:3300/0 conn(0x7fe0740b3990 0x7fe0740af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 -- 172.21.15.19:0/3197356778 >> v1:172.21.15.19:6789/0 conn(0x7fe0740b3d60 legacy=0x7fe074141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:21.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient: _finish_hunting 0 2023-10-06T13:53:21.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 monclient: found mon.noname-c-legacy 2023-10-06T13:53:21.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:21.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.83:6789/0 2023-10-06T13:53:21.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 -- 172.21.15.19:0/3197356778 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe074141c10 con 0x7fe0740b00c0 2023-10-06T13:53:21.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient: _finish_auth 0 2023-10-06T13:53:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient: _check_auth_tickets 2023-10-06T13:53:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 -- 172.21.15.19:0/3197356778 <== mon.1 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251531364 0 0) 0x7fe06c0019f0 con 0x7fe0740b00c0 2023-10-06T13:53:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2023-10-06T13:53:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient: dump: 2023-10-06T13:53:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.945 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] but i'm connected to v1:172.21.15.83:6789/0, reconnecting 2023-10-06T13:53:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 -- 172.21.15.19:0/3197356778 >> v1:172.21.15.83:6789/0 conn(0x7fe0740b00c0 legacy=0x7fe07414b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:53:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 --2- 172.21.15.19:0/3197356778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe05c003970 0x7fe05c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient(hunting): picked mon.a con 0x7fe05c003970 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 --2- 172.21.15.19:0/3197356778 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe05c0047c0 0x7fe05c004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient(hunting): picked mon.c con 0x7fe05c0047c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 --2- 172.21.15.19:0/3197356778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe05c0052c0 0x7fe05c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient(hunting): picked mon.b con 0x7fe05c0052c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 -- 172.21.15.19:0/3197356778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe074140dd0 con 0x7fe05c003970 2023-10-06T13:53:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 -- 172.21.15.19:0/3197356778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c00c1f0 con 0x7fe05c0052c0 2023-10-06T13:53:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 1 -- 172.21.15.19:0/3197356778 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c00c3b0 con 0x7fe05c0047c0 2023-10-06T13:53:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.940+0000 7fe0727fc640 10 monclient(hunting): _finish_auth 0 2023-10-06T13:53:21.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.941+0000 7fe072ffd640 1 --2- 172.21.15.19:0/3197356778 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe05c0047c0 0x7fe05c004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.942+0000 7fe072ffd640 10 monclient(hunting): get_auth_request con 0x7fe05c0047c0 auth_method 0 2023-10-06T13:53:21.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.942+0000 7fe072ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.942+0000 7fe072ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.942+0000 7fe072ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:53:21.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.942+0000 7fe073fff640 1 --2- 172.21.15.19:0/3197356778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe05c0052c0 0x7fe05c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.942+0000 7fe073fff640 10 monclient(hunting): get_auth_request con 0x7fe05c0052c0 auth_method 0 2023-10-06T13:53:21.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.942+0000 7fe073fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.942+0000 7fe073fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.942+0000 7fe073fff640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:53:21.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.943+0000 7fe072ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.943+0000 7fe072ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.943+0000 7fe072ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T13:53:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.943+0000 7fe073fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.943+0000 7fe073fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.943+0000 7fe073fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T13:53:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.944+0000 7fe073fff640 10 monclient(hunting): handle_auth_done global_id 4366 payload 293 2023-10-06T13:53:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.944+0000 7fe073fff640 1 -- 172.21.15.19:0/3197356778 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe05c0047c0 msgr2=0x7fe05c004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.944+0000 7fe073fff640 1 --2- 172.21.15.19:0/3197356778 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe05c0047c0 0x7fe05c004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.944+0000 7fe073fff640 1 -- 172.21.15.19:0/3197356778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe05c003970 msgr2=0x7fe05c003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.944+0000 7fe073fff640 1 --2- 172.21.15.19:0/3197356778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe05c003970 0x7fe05c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.944+0000 7fe073fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.944+0000 7fe073fff640 1 monclient: found mon.b 2023-10-06T13:53:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.944+0000 7fe073fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.944+0000 7fe073fff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.944+0000 7fe073fff640 1 -- 172.21.15.19:0/3197356778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe074141c10 con 0x7fe05c0052c0 2023-10-06T13:53:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.944+0000 7fe073fff640 10 monclient: _finish_auth 0 2023-10-06T13:53:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.944+0000 7fe073fff640 10 monclient: _check_auth_tickets 2023-10-06T13:53:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.944+0000 7fe073fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.945+0000 7fe07a0b6640 5 monclient: authenticate success, global_id 4366 2023-10-06T13:53:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.945+0000 7fe07a0b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.945+0000 7fe073fff640 1 -- 172.21.15.19:0/3197356778 reap_dead start 2023-10-06T13:53:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.947+0000 7fe073fff640 1 --2- 172.21.15.19:0/3197356778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe05c0052c0 0x7fe05c00b980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00a8b0 tx=0x7fe05c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6afa9ba32a54e96 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:21.952 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.950+0000 7fe0727fc640 1 -- 172.21.15.19:0/3197356778 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c034630 con 0x7fe05c0052c0 2023-10-06T13:53:21.952 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.950+0000 7fe0727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.952 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.950+0000 7fe0727fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:21.952 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.950+0000 7fe0727fc640 10 monclient: dump: 2023-10-06T13:53:21.952 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.952 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.952 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.952 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.953 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.953 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.953 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.953 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.953 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.953 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.950+0000 7fe0727fc640 1 -- 172.21.15.19:0/3197356778 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c02f070 con 0x7fe05c0052c0 2023-10-06T13:53:21.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.950+0000 7fe07a0b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:21.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.950+0000 7fe0727fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:21.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.950+0000 7fe07a0b6640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:21.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.950+0000 7fe0727fc640 1 -- 172.21.15.19:0/3197356778 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c034a10 con 0x7fe05c0052c0 2023-10-06T13:53:21.953 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.950+0000 7fe07a0b6640 4 set_mon_vals no callback set 2023-10-06T13:53:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.950+0000 7fe0727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.950+0000 7fe0727fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.950+0000 7fe0727fc640 10 monclient: dump: 2023-10-06T13:53:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.954 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.951+0000 7fe07a0b6640 10 monclient: shutdown 2023-10-06T13:53:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.951+0000 7fe07a0b6640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.951+0000 7fe07a0b6640 1 -- 172.21.15.19:0/3197356778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe05c0052c0 msgr2=0x7fe05c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.951+0000 7fe07a0b6640 1 --2- 172.21.15.19:0/3197356778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe05c0052c0 0x7fe05c00b980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00a8b0 tx=0x7fe05c0057f0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.951+0000 7fe07a0b6640 1 -- 172.21.15.19:0/3197356778 shutdown_connections 2023-10-06T13:53:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.951+0000 7fe07a0b6640 1 --2- 172.21.15.19:0/3197356778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe05c0052c0 0x7fe05c00b980 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.951+0000 7fe07a0b6640 1 -- 172.21.15.19:0/3197356778 >> 172.21.15.19:0/3197356778 conn(0x7fe0740b6e00 msgr2=0x7fe0740b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.951+0000 7fe07a0b6640 1 -- 172.21.15.19:0/3197356778 shutdown_connections 2023-10-06T13:53:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.951+0000 7fe07a0b6640 1 -- 172.21.15.19:0/3197356778 wait complete. 2023-10-06T13:53:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.952+0000 7fe07a0b6640 10 monclient: build_initial_monmap 2023-10-06T13:53:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.952+0000 7fe07a0b6640 10 monclient: monmap: 2023-10-06T13:53:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:21.959 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.952+0000 7fe07a0b6640 1 librados: starting msgr at 2023-10-06T13:53:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.952+0000 7fe07a0b6640 1 librados: starting objecter 2023-10-06T13:53:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.953+0000 7fe07a0b6640 1 Processor -- start 2023-10-06T13:53:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.953+0000 7fe07a0b6640 1 -- start start 2023-10-06T13:53:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.953+0000 7fe07a0b6640 1 librados: setting wanted keys 2023-10-06T13:53:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.953+0000 7fe07a0b6640 1 librados: calling monclient init 2023-10-06T13:53:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.953+0000 7fe07a0b6640 10 monclient: init 2023-10-06T13:53:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T13:53:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe05c0052c0 0x7fe0740a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 10 monclient(hunting): picked mon.noname-b con 0x7fe05c0052c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe0740aaea0 0x7fe0740ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 10 monclient(hunting): picked mon.noname-c con 0x7fe0740aaea0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0740aca30 0x7fe0741d5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 10 monclient(hunting): picked mon.noname-a con 0x7fe0740aca30 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0740a9b00 con 0x7fe0740aca30 2023-10-06T13:53:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0740aa980 con 0x7fe05c0052c0 2023-10-06T13:53:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe074144b70 con 0x7fe0740aaea0 2023-10-06T13:53:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe07a0b6640 10 monclient(hunting): authenticate will time out at 1330.140689s 2023-10-06T13:53:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe0737fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe05c0052c0 0x7fe0740a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.954+0000 7fe072ffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe0740aaea0 0x7fe0740ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.955+0000 7fe0737fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe05c0052c0 0x7fe0740a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39562/0 (socket says 172.21.15.19:39562) 2023-10-06T13:53:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.955+0000 7fe0737fe640 1 -- 172.21.15.19:0/694664086 learned_addr learned my addr 172.21.15.19:0/694664086 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.955+0000 7fe0737fe640 10 monclient(hunting): get_auth_request con 0x7fe05c0052c0 auth_method 0 2023-10-06T13:53:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.955+0000 7fe0737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.955+0000 7fe0737fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.955+0000 7fe0737fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.955+0000 7fe072ffd640 10 monclient(hunting): get_auth_request con 0x7fe0740aaea0 auth_method 0 2023-10-06T13:53:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.955+0000 7fe072ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.955+0000 7fe072ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.955+0000 7fe072ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.955+0000 7fe072ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.955+0000 7fe072ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.955+0000 7fe072ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.957+0000 7fe0737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.957+0000 7fe0737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.957+0000 7fe0737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.959+0000 7fe0737fe640 10 monclient(hunting): handle_auth_done global_id 4369 payload 824 2023-10-06T13:53:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.959+0000 7fe0737fe640 1 -- 172.21.15.19:0/694664086 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe0740aaea0 msgr2=0x7fe0740ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.959+0000 7fe0737fe640 1 --2- 172.21.15.19:0/694664086 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe0740aaea0 0x7fe0740ac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.959+0000 7fe0737fe640 1 -- 172.21.15.19:0/694664086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0740aca30 msgr2=0x7fe0741d5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.959+0000 7fe0737fe640 1 --2- 172.21.15.19:0/694664086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0740aca30 0x7fe0741d5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.959+0000 7fe0737fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.959+0000 7fe0737fe640 1 monclient: found mon.noname-b 2023-10-06T13:53:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.959+0000 7fe0737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.959+0000 7fe0737fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.959+0000 7fe0737fe640 1 -- 172.21.15.19:0/694664086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0740acea0 con 0x7fe05c0052c0 2023-10-06T13:53:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.962+0000 7fe0737fe640 1 --2- 172.21.15.19:0/694664086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe05c0052c0 0x7fe0740a9f70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe06400b500 tx=0x7fe06400b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.962+0000 7fe05bfff640 10 client.?.objecter ms_handle_connect 0x7fe05c0052c0 2023-10-06T13:53:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.962+0000 7fe05bfff640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe05bfff640 1 -- 172.21.15.19:0/694664086 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064010870 con 0x7fe05c0052c0 2023-10-06T13:53:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe05bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe05bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe05bfff640 10 monclient: dump: 2023-10-06T13:53:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.970 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe05bfff640 10 monclient: _finish_auth 0 2023-10-06T13:53:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe05bfff640 10 monclient: _check_auth_tickets 2023-10-06T13:53:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe05bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe05bfff640 1 -- 172.21.15.19:0/694664086 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0640109d0 con 0x7fe05c0052c0 2023-10-06T13:53:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe07a0b6640 5 monclient: authenticate success, global_id 4369 2023-10-06T13:53:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe07a0b6640 10 monclient: _renew_subs 2023-10-06T13:53:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe07a0b6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe07a0b6640 1 -- 172.21.15.19:0/694664086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0740aa5a0 con 0x7fe05c0052c0 2023-10-06T13:53:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe05bfff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe05bfff640 1 -- 172.21.15.19:0/694664086 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064010d80 con 0x7fe05c0052c0 2023-10-06T13:53:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe05bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe05bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe05bfff640 10 monclient: dump: 2023-10-06T13:53:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:21.972 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe071ffb640 4 set_mon_vals no callback set 2023-10-06T13:53:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe07a0b6640 10 client.4369.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe07a0b6640 10 monclient: _renew_subs 2023-10-06T13:53:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe07a0b6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.966+0000 7fe07a0b6640 1 -- 172.21.15.19:0/694664086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe074144440 con 0x7fe05c0052c0 2023-10-06T13:53:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.967+0000 7fe07a0b6640 1 librados: init done 2023-10-06T13:53:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.967+0000 7fe07a0b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.967+0000 7fe07a0b6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.967+0000 7fe07a0b6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.967+0000 7fe07a0b6640 1 -- 172.21.15.19:0/694664086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0740678e0 con 0x7fe05c0052c0 2023-10-06T13:53:21.988 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.986+0000 7fe05bfff640 1 -- 172.21.15.19:0/694664086 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe064023480 con 0x7fe05c0052c0 2023-10-06T13:53:21.989 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.986+0000 7fe05bfff640 1 --2- 172.21.15.19:0/694664086 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe06006c1c0 0x7fe06006e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:21.989 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.987+0000 7fe072ffd640 1 --2- 172.21.15.19:0/694664086 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe06006c1c0 0x7fe06006e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:21.989 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.987+0000 7fe072ffd640 10 monclient: get_auth_request con 0x7fe06006c1c0 auth_method 0 2023-10-06T13:53:21.989 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.987+0000 7fe072ffd640 1 --2- 172.21.15.19:0/694664086 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe06006c1c0 0x7fe06006e670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe0740ab760 tx=0x7fe068009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:21.989 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.987+0000 7fe05bfff640 10 client.4369.objecter ms_handle_connect 0x7fe06006c1c0 2023-10-06T13:53:21.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.989+0000 7fe05bfff640 1 -- 172.21.15.19:0/694664086 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe064058270 con 0x7fe05c0052c0 2023-10-06T13:53:21.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.989+0000 7fe05bfff640 10 client.4369.objecter ms_dispatch 0x7fe074000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:21.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.989+0000 7fe05bfff640 3 client.4369.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:21.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.989+0000 7fe05bfff640 3 client.4369.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:21.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:21.989+0000 7fe05bfff640 20 client.4369.objecter dump_active .. 0 homeless 2023-10-06T13:53:22.036 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.034+0000 7fc40eafc640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2023-10-06T13:53:22.036 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.034+0000 7fc40eafc640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2023-10-06T13:53:22.036 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.034+0000 7fc40eafc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:22.036 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.034+0000 7fc40eafc640 1 -- 172.21.15.19:0/2428521550 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc4080678e0 con 0x7fc4080b3990 2023-10-06T13:53:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.104+0000 7f48f7fff640 1 -- 172.21.15.19:0/4068037190 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4900079390 con 0x7f48f8003970 2023-10-06T13:53:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.104+0000 7f48f7fff640 10 client.4341.objecter ms_dispatch 0x7f4910000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.104+0000 7f48f7fff640 3 client.4341.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.104+0000 7f48f7fff640 3 client.4341.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.104+0000 7f48f7fff640 20 client.4341.objecter dump_active .. 0 homeless 2023-10-06T13:53:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.105+0000 7f48f7fff640 1 -- 172.21.15.19:0/4068037190 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f4900074fd0 con 0x7f48f8003970 2023-10-06T13:53:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.105+0000 7f48f7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:22.108 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.105+0000 7f48f7fff640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:22.155 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.153+0000 7f7334ecf640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2023-10-06T13:53:22.155 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.153+0000 7f7334ecf640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2023-10-06T13:53:22.155 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.153+0000 7f7334ecf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:22.155 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.153+0000 7f7334ecf640 1 -- 172.21.15.19:0/4091043265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f73300ae200 con 0x7f73300b00c0 2023-10-06T13:53:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.186+0000 7f9d15d9b640 1 -- 172.21.15.19:0/3002850128 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0802bd60 con 0x7f9d00003970 2023-10-06T13:53:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.186+0000 7f9d15d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.186+0000 7f9d15d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.186+0000 7f9d15d9b640 10 monclient: dump: 2023-10-06T13:53:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.189 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.186+0000 7f9d1d246640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.187+0000 7fc4057fa640 1 -- 172.21.15.19:0/2428521550 <== mon.2 v2:172.21.15.19:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc3fc017090 con 0x7fc4080b3990 2023-10-06T13:53:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.187+0000 7fc4057fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2023-10-06T13:53:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.187+0000 7fc4057fa640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:22.190 INFO:teuthology.orchestra.run.smithi019.stdout:64424509443 2023-10-06T13:53:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.189+0000 7fc40eafc640 10 librados: watch_flush enter 2023-10-06T13:53:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.189+0000 7fc40eafc640 10 librados: watch_flush exit 2023-10-06T13:53:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.189+0000 7fc40eafc640 20 client.4277.objecter shutdown clearing up homeless session... 2023-10-06T13:53:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.189+0000 7fc40eafc640 10 client.4277.objecter successfully canceled tick 2023-10-06T13:53:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 1 -- 172.21.15.19:0/2428521550 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc3f406c210 msgr2=0x7fc3f406e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 1 --2- 172.21.15.19:0/2428521550 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc3f406c210 0x7fc3f406e6c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc3f802b8e0 tx=0x7fc3f803c040 comp rx=0 tx=0).stop 2023-10-06T13:53:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 10 monclient: shutdown 2023-10-06T13:53:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 1 -- 172.21.15.19:0/2428521550 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc4080b3990 msgr2=0x7fc4080ac6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 1 --2- 172.21.15.19:0/2428521550 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc4080b3990 0x7fc4080ac6e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00c390 tx=0x7fc3fc00c850 comp rx=0 tx=0).stop 2023-10-06T13:53:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 1 -- 172.21.15.19:0/2428521550 shutdown_connections 2023-10-06T13:53:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 1 --2- 172.21.15.19:0/2428521550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4080b3d60 0x7fc4081d6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 1 --2- 172.21.15.19:0/2428521550 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc3f406c210 0x7fc3f406e6c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 1 --2- 172.21.15.19:0/2428521550 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc4080b3990 0x7fc4080ac6e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 1 --2- 172.21.15.19:0/2428521550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4080b00c0 0x7fc4080abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 1 -- 172.21.15.19:0/2428521550 >> 172.21.15.19:0/2428521550 conn(0x7fc4080b6e00 msgr2=0x7fc40806b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 1 -- 172.21.15.19:0/2428521550 shutdown_connections 2023-10-06T13:53:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 1 -- 172.21.15.19:0/2428521550 wait complete. 2023-10-06T13:53:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.190+0000 7fc40eafc640 1 librados: shutdown 2023-10-06T13:53:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.192+0000 7f9d15d9b640 1 -- 172.21.15.19:0/3002850128 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d08033440 con 0x7f9d00003970 2023-10-06T13:53:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.192+0000 7f9d15d9b640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.192+0000 7f9d15d9b640 1 -- 172.21.15.19:0/3002850128 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d080337f0 con 0x7f9d00003970 2023-10-06T13:53:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.192+0000 7f9d15d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.192+0000 7f9d15d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.192+0000 7f9d15d9b640 10 monclient: dump: 2023-10-06T13:53:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.195 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.192+0000 7f9d1d246640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.192+0000 7f9d1d246640 4 set_mon_vals no callback set 2023-10-06T13:53:22.196 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.193+0000 7f9d1d246640 10 monclient: shutdown 2023-10-06T13:53:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.193+0000 7f9d1d246640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.193+0000 7f9d1d246640 1 -- 172.21.15.19:0/3002850128 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9d00003970 msgr2=0x7f9d00003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.193+0000 7f9d1d246640 1 --2- 172.21.15.19:0/3002850128 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9d00003970 0x7f9d00003d60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9d1804f900 tx=0x7f9d000057f0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.193+0000 7f9d1d246640 1 -- 172.21.15.19:0/3002850128 shutdown_connections 2023-10-06T13:53:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.193+0000 7f9d1d246640 1 --2- 172.21.15.19:0/3002850128 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9d00003970 0x7f9d00003d60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.193+0000 7f9d1d246640 1 -- 172.21.15.19:0/3002850128 >> 172.21.15.19:0/3002850128 conn(0x7f9d180b6e00 msgr2=0x7f9d180b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.193+0000 7f9d1d246640 1 -- 172.21.15.19:0/3002850128 shutdown_connections 2023-10-06T13:53:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.193+0000 7f9d1d246640 1 -- 172.21.15.19:0/3002850128 wait complete. 2023-10-06T13:53:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.194+0000 7f9d1d246640 10 monclient: build_initial_monmap 2023-10-06T13:53:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.194+0000 7f9d1d246640 10 monclient: monmap: 2023-10-06T13:53:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:22.198 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:22.198 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:22.198 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:22.198 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:22.198 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.198 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:22.198 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:22.198 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:22.198 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.198 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.194+0000 7f9d1d246640 1 librados: starting msgr at 2023-10-06T13:53:22.198 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.194+0000 7f9d1d246640 1 librados: starting objecter 2023-10-06T13:53:22.198 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.195+0000 7f9d1d246640 1 Processor -- start 2023-10-06T13:53:22.198 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.195+0000 7f9d1d246640 1 -- start start 2023-10-06T13:53:22.199 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.195+0000 7f9d1d246640 1 librados: setting wanted keys 2023-10-06T13:53:22.199 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.195+0000 7f9d1d246640 1 librados: calling monclient init 2023-10-06T13:53:22.199 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.195+0000 7f9d1d246640 10 monclient: init 2023-10-06T13:53:22.199 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.195+0000 7f9d1d246640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.195+0000 7f9d1d246640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T13:53:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.195+0000 7f9d1d246640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d00003970 0x7f9d181d9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.195+0000 7f9d1d246640 10 monclient(hunting): picked mon.noname-b con 0x7f9d00003970 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.195+0000 7f9d1d246640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d180ac020 0x7f9d181d9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.195+0000 7f9d1d246640 10 monclient(hunting): picked mon.noname-a con 0x7f9d180ac020 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.195+0000 7f9d1d246640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9d180a9ef0 0x7f9d180aa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.196+0000 7f9d1d246640 10 monclient(hunting): picked mon.noname-c con 0x7f9d180a9ef0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.196+0000 7f9d1d246640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.196+0000 7f9d1d246640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d181d9170 con 0x7f9d180ac020 2023-10-06T13:53:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.196+0000 7f9d1d246640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.196+0000 7f9d1d246640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180a8960 con 0x7f9d00003970 2023-10-06T13:53:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.196+0000 7f9d1d246640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.201 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.196+0000 7f9d1d246640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180a8ad0 con 0x7f9d180a9ef0 2023-10-06T13:53:22.201 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.196+0000 7f9d1d246640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:22.201 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.196+0000 7f9d1d246640 10 monclient(hunting): authenticate will time out at 1330.382430s 2023-10-06T13:53:22.201 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.198+0000 7f9d1659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d180ac020 0x7f9d181d9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.201 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.198+0000 7f9d1659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d180ac020 0x7f9d181d9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39066/0 (socket says 172.21.15.19:39066) 2023-10-06T13:53:22.201 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.198+0000 7f9d1659c640 1 -- 172.21.15.19:0/4111966358 learned_addr learned my addr 172.21.15.19:0/4111966358 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:22.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.198+0000 7f9d1659c640 10 monclient(hunting): get_auth_request con 0x7f9d180ac020 auth_method 0 2023-10-06T13:53:22.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.198+0000 7f9d1659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:22.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.198+0000 7f9d1659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:22.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.198+0000 7f9d1659c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:22.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.199+0000 7f9d1659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:22.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.199+0000 7f9d1659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:22.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.199+0000 7f9d1659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:22.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.200+0000 7f9d1759e640 1 --2- 172.21.15.19:0/4111966358 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9d180a9ef0 0x7f9d180aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.201+0000 7f9d1759e640 10 monclient(hunting): get_auth_request con 0x7f9d180a9ef0 auth_method 0 2023-10-06T13:53:22.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.201+0000 7f9d1759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:22.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.201+0000 7f9d1759e640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:22.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.201+0000 7f9d1759e640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:22.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.202+0000 7f9d1759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:22.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.202+0000 7f9d1759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:22.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.202+0000 7f9d1759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:22.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.202+0000 7f9d1659c640 10 monclient(hunting): handle_auth_done global_id 4344 payload 824 2023-10-06T13:53:22.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.202+0000 7f9d1659c640 1 -- 172.21.15.19:0/4111966358 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9d180a9ef0 msgr2=0x7f9d180aa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.202+0000 7f9d1659c640 1 --2- 172.21.15.19:0/4111966358 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9d180a9ef0 0x7f9d180aa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.202+0000 7f9d1659c640 1 -- 172.21.15.19:0/4111966358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d00003970 msgr2=0x7f9d181d9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.205 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.202+0000 7f9d1659c640 1 --2- 172.21.15.19:0/4111966358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d00003970 0x7f9d181d9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.205 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.202+0000 7f9d1659c640 10 monclient: _finish_hunting 0 2023-10-06T13:53:22.205 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.202+0000 7f9d1659c640 1 monclient: found mon.noname-a 2023-10-06T13:53:22.205 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.202+0000 7f9d1659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:22.205 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.202+0000 7f9d1659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:22.205 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.202+0000 7f9d1659c640 1 -- 172.21.15.19:0/4111966358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d180aa8a0 con 0x7f9d180ac020 2023-10-06T13:53:22.209 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.2 2023-10-06T13:53:22.209 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.207+0000 7f9d1659c640 1 --2- 172.21.15.19:0/4111966358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d180ac020 0x7f9d181d9b90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00c710 tx=0x7f9d0c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.207+0000 7f9cff7fe640 10 client.?.objecter ms_handle_connect 0x7f9d180ac020 2023-10-06T13:53:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.207+0000 7f9cff7fe640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9cff7fe640 1 -- 172.21.15.19:0/4111966358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0074e0 con 0x7f9d180ac020 2023-10-06T13:53:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9cff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9cff7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9cff7fe640 10 monclient: dump: 2023-10-06T13:53:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.215 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9cff7fe640 10 monclient: _finish_auth 0 2023-10-06T13:53:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9cff7fe640 10 monclient: _check_auth_tickets 2023-10-06T13:53:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9cff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9cff7fe640 1 -- 172.21.15.19:0/4111966358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c007640 con 0x7f9d180ac020 2023-10-06T13:53:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9d1d246640 5 monclient: authenticate success, global_id 4344 2023-10-06T13:53:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9cff7fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9d1d246640 10 monclient: _renew_subs 2023-10-06T13:53:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9cff7fe640 1 -- 172.21.15.19:0/4111966358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c004a20 con 0x7f9d180ac020 2023-10-06T13:53:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9d1d246640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9d1d246640 1 -- 172.21.15.19:0/4111966358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d180aab50 con 0x7f9d180ac020 2023-10-06T13:53:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9cff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9cff7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9cff7fe640 10 monclient: dump: 2023-10-06T13:53:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.217 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9d1559a640 4 set_mon_vals no callback set 2023-10-06T13:53:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9d1d246640 10 client.4344.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9d1d246640 10 monclient: _renew_subs 2023-10-06T13:53:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9d1d246640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9d1d246640 1 -- 172.21.15.19:0/4111966358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d181d5d00 con 0x7f9d180ac020 2023-10-06T13:53:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.212+0000 7f9d1d246640 1 librados: init done 2023-10-06T13:53:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.213+0000 7f9d1d246640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.213+0000 7f9d1d246640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.213+0000 7f9d1d246640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.213+0000 7f9d1d246640 1 -- 172.21.15.19:0/4111966358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d180678e0 con 0x7f9d180ac020 2023-10-06T13:53:22.224 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.223+0000 7fe05bfff640 1 -- 172.21.15.19:0/694664086 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7fe0640545b0 con 0x7fe05c0052c0 2023-10-06T13:53:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.223+0000 7fe05bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.223+0000 7fe05bfff640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:22.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.232+0000 7f731effd640 1 -- 172.21.15.19:0/4091043265 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7320079d20 con 0x7f73300b00c0 2023-10-06T13:53:22.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.232+0000 7f731effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2023-10-06T13:53:22.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.232+0000 7f731effd640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:22.234 INFO:teuthology.orchestra.run.smithi019.stdout:68719476741 2023-10-06T13:53:22.237 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 10 librados: watch_flush enter 2023-10-06T13:53:22.238 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 10 librados: watch_flush exit 2023-10-06T13:53:22.238 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 20 client.4360.objecter shutdown clearing up homeless session... 2023-10-06T13:53:22.238 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 10 client.4360.objecter successfully canceled tick 2023-10-06T13:53:22.238 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 1 -- 172.21.15.19:0/4091043265 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f731006c210 msgr2=0x7f731006e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.238 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 1 --2- 172.21.15.19:0/4091043265 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f731006c210 0x7f731006e6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f733013eaa0 tx=0x7f7324009040 comp rx=0 tx=0).stop 2023-10-06T13:53:22.238 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 10 monclient: shutdown 2023-10-06T13:53:22.238 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:22.238 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 1 -- 172.21.15.19:0/4091043265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73300b00c0 msgr2=0x7f73301427d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.238 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 1 --2- 172.21.15.19:0/4091043265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73300b00c0 0x7f73301427d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7320034040 tx=0x7f73200046c0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 1 -- 172.21.15.19:0/4091043265 shutdown_connections 2023-10-06T13:53:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 1 --2- 172.21.15.19:0/4091043265 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f731006c210 0x7f731006e6c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 1 --2- 172.21.15.19:0/4091043265 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f73300b3d60 0x7f7330135f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 1 --2- 172.21.15.19:0/4091043265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73300b3990 0x7f7330139060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 1 --2- 172.21.15.19:0/4091043265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73300b00c0 0x7f73301427d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 1 -- 172.21.15.19:0/4091043265 >> 172.21.15.19:0/4091043265 conn(0x7f73300b6e00 msgr2=0x7f73300b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 1 -- 172.21.15.19:0/4091043265 shutdown_connections 2023-10-06T13:53:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.235+0000 7f7334ecf640 1 -- 172.21.15.19:0/4091043265 wait complete. 2023-10-06T13:53:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.236+0000 7f7334ecf640 1 librados: shutdown 2023-10-06T13:53:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.239+0000 7f9cff7fe640 1 -- 172.21.15.19:0/4111966358 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f9d0c00a5e0 con 0x7f9d180ac020 2023-10-06T13:53:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.240+0000 7f9cff7fe640 1 --2- 172.21.15.19:0/4111966358 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9d0406c1c0 0x7f9d0406e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.240+0000 7f9d16d9d640 1 --2- 172.21.15.19:0/4111966358 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9d0406c1c0 0x7f9d0406e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.240+0000 7f9d16d9d640 10 monclient: get_auth_request con 0x7f9d0406c1c0 auth_method 0 2023-10-06T13:53:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.241+0000 7f9cff7fe640 10 client.4344.objecter ms_handle_connect 0x7f9d0406c1c0 2023-10-06T13:53:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.241+0000 7f9d16d9d640 1 --2- 172.21.15.19:0/4111966358 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9d0406c1c0 0x7f9d0406e670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9d08035000 tx=0x7f9d0802d070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:22.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476741 for osd.1 2023-10-06T13:53:22.298 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T13:53:22.298 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T13:53:22.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.312+0000 7f491589d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2023-10-06T13:53:22.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.312+0000 7f491589d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2023-10-06T13:53:22.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.312+0000 7f491589d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:22.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.312+0000 7f491589d640 1 -- 172.21.15.19:0/4068037190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f49100ad3a0 con 0x7f48f8003970 2023-10-06T13:53:22.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T13:53:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.402+0000 7f9cff7fe640 1 -- 172.21.15.19:0/4111966358 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9d0c00ce20 con 0x7f9d180ac020 2023-10-06T13:53:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.402+0000 7f9cff7fe640 10 client.4344.objecter ms_dispatch 0x7f9d18000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.402+0000 7f9cff7fe640 3 client.4344.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.402+0000 7f9cff7fe640 3 client.4344.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.402+0000 7f9cff7fe640 20 client.4344.objecter dump_active .. 0 homeless 2023-10-06T13:53:22.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.441+0000 7fe049ae2640 10 monclient: get_monmap_and_config 2023-10-06T13:53:22.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.441+0000 7fe049ae2640 10 monclient: build_initial_monmap 2023-10-06T13:53:22.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.441+0000 7fe049ae2640 10 monclient: monmap: 2023-10-06T13:53:22.443 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:22.443 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.444 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:22.442413+0000 2023-10-06T13:53:22.444 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:22.442413+0000 2023-10-06T13:53:22.444 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:22.444 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.444 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:22.444 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:22.444 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:22.444 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:22.444 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:22.444 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.445 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.442+0000 7fe049ae2640 1 Processor -- start 2023-10-06T13:53:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.442+0000 7fe049ae2640 1 -- start start 2023-10-06T13:53:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.442+0000 7fe049ae2640 10 monclient: init 2023-10-06T13:53:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7fe049ae2640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7fe049ae2640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2023-10-06T13:53:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7fe049ae2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe0440b3990 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7fe049ae2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe0440b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7fe049ae2640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe0440b07d0 0x7fe0440af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7fe049ae2640 10 monclient(hunting): picked mon.noname-a con 0x7fe0440b07d0 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7fe049ae2640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0440aebe0 con 0x7fe0440b3990 2023-10-06T13:53:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7fe049ae2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe044141070 con 0x7fe0440b3d60 2023-10-06T13:53:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7fe049ae2640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.447 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7fe049ae2640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe044140dd0 con 0x7fe0440b07d0 2023-10-06T13:53:22.447 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7fe049ae2640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:22.447 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7fe049ae2640 10 monclient(hunting): authenticate will time out at 1330.629721s 2023-10-06T13:53:22.447 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7fe042ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe0440b3d60 0x7fe0440b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:38414/0 (socket says 172.21.15.19:38414) 2023-10-06T13:53:22.447 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7fe042ffd640 1 -- 172.21.15.19:0/102371369 learned_addr learned my addr 172.21.15.19:0/102371369 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7f88351ca640 10 monclient: get_monmap_and_config 2023-10-06T13:53:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7f88351ca640 10 monclient: build_initial_monmap 2023-10-06T13:53:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.443+0000 7f88351ca640 10 monclient: monmap: 2023-10-06T13:53:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:22.444981+0000 2023-10-06T13:53:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:22.444981+0000 2023-10-06T13:53:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:22.451 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.444+0000 7f88351ca640 1 Processor -- start 2023-10-06T13:53:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.444+0000 7f88351ca640 1 -- start start 2023-10-06T13:53:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.444+0000 7f88351ca640 10 monclient: init 2023-10-06T13:53:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.445+0000 7f88351ca640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.445+0000 7f88351ca640 10 monclient: _add_conns ranks=[1,3,4,2,0] 2023-10-06T13:53:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.445+0000 7f88351ca640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f88300b3990 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.445+0000 7f88351ca640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f88300b3d60 0x7f88300af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.445+0000 7f88351ca640 10 monclient(hunting): picked mon.noname-a con 0x7f88300b3d60 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.445+0000 7f88351ca640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88300b00c0 0x7f88300b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.445+0000 7f88351ca640 10 monclient(hunting): picked mon.noname-c con 0x7f88300b00c0 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.445+0000 7f88351ca640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88300aebe0 con 0x7f88300b3990 2023-10-06T13:53:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.445+0000 7f88351ca640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.445+0000 7f88351ca640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88300b0a60 con 0x7f88300b3d60 2023-10-06T13:53:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.445+0000 7f88351ca640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.445+0000 7f88351ca640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8830140dd0 con 0x7f88300b00c0 2023-10-06T13:53:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.445+0000 7f88351ca640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.445+0000 7f88351ca640 10 monclient(hunting): authenticate will time out at 1330.632245s 2023-10-06T13:53:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.446+0000 7f882f59e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88300b00c0 0x7f88300b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.447+0000 7f882f59e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88300b00c0 0x7f88300b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39570/0 (socket says 172.21.15.19:39570) 2023-10-06T13:53:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.447+0000 7f882f59e640 1 -- 172.21.15.19:0/1426498120 learned_addr learned my addr 172.21.15.19:0/1426498120 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.447+0000 7f882f59e640 10 monclient(hunting): get_auth_request con 0x7f88300b00c0 auth_method 0 2023-10-06T13:53:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.447+0000 7f882f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.447+0000 7f882f59e640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.447+0000 7f882f59e640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.448+0000 7f882f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.448+0000 7f882f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.448+0000 7f882f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.450+0000 7f882f59e640 10 monclient(hunting): handle_auth_done global_id 4372 payload 274 2023-10-06T13:53:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.450+0000 7f882f59e640 1 -- 172.21.15.19:0/1426498120 >> v2:172.21.15.19:3300/0 conn(0x7f88300b3d60 msgr2=0x7f88300af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.450+0000 7f882f59e640 1 --2- 172.21.15.19:0/1426498120 >> v2:172.21.15.19:3300/0 conn(0x7f88300b3d60 0x7f88300af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.450+0000 7f882f59e640 1 -- 172.21.15.19:0/1426498120 >> v1:172.21.15.83:6789/0 conn(0x7f88300b3990 legacy=0x7f8830141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.450+0000 7f882f59e640 10 monclient: _finish_hunting 0 2023-10-06T13:53:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.450+0000 7f882f59e640 1 monclient: found mon.noname-c 2023-10-06T13:53:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.450+0000 7f882f59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.450+0000 7f882f59e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T13:53:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.450+0000 7f882f59e640 1 -- 172.21.15.19:0/1426498120 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8830141070 con 0x7f88300b00c0 2023-10-06T13:53:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.452+0000 7fe07a0b6640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2023-10-06T13:53:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.452+0000 7fe07a0b6640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2023-10-06T13:53:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.452+0000 7fe07a0b6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:22.456 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.452+0000 7fe07a0b6640 1 -- 172.21.15.19:0/694664086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe074144440 con 0x7fe05c0052c0 2023-10-06T13:53:22.456 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.452+0000 7f882f59e640 1 --2- 172.21.15.19:0/1426498120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88300b00c0 0x7f88300b0520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f88300ae820 tx=0x7f882802bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=650e911baa78b45a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.459+0000 7fe05bfff640 1 -- 172.21.15.19:0/694664086 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe074144440 con 0x7fe05c0052c0 2023-10-06T13:53:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.459+0000 7fe05bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2023-10-06T13:53:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.459+0000 7fe05bfff640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:22.461 INFO:teuthology.orchestra.run.smithi019.stdout:60129542148 2023-10-06T13:53:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f882e59c640 1 -- 172.21.15.19:0/1426498120 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882802ca80 con 0x7f88300b00c0 2023-10-06T13:53:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f882e59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f882e59c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f882e59c640 10 monclient: dump: 2023-10-06T13:53:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.464 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f882e59c640 10 monclient: _finish_auth 0 2023-10-06T13:53:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f882e59c640 10 monclient: _check_auth_tickets 2023-10-06T13:53:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f882e59c640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f882e59c640 1 -- 172.21.15.19:0/1426498120 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882802cbe0 con 0x7f88300b00c0 2023-10-06T13:53:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f882e59c640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f88351ca640 5 monclient: authenticate success, global_id 4372 2023-10-06T13:53:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f882e59c640 1 -- 172.21.15.19:0/1426498120 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88280343f0 con 0x7f88300b00c0 2023-10-06T13:53:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f882e59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f882e59c640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f882e59c640 10 monclient: dump: 2023-10-06T13:53:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.465 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f88351ca640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.461+0000 7f88351ca640 4 set_mon_vals no callback set 2023-10-06T13:53:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7f88351ca640 10 monclient: shutdown 2023-10-06T13:53:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7f88351ca640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7f88351ca640 1 -- 172.21.15.19:0/1426498120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88300b00c0 msgr2=0x7f88300b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7f88351ca640 1 --2- 172.21.15.19:0/1426498120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88300b00c0 0x7f88300b0520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f88300ae820 tx=0x7f882802bee0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7f88351ca640 1 -- 172.21.15.19:0/1426498120 shutdown_connections 2023-10-06T13:53:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7f88351ca640 1 --2- 172.21.15.19:0/1426498120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88300b00c0 0x7f88300b0520 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7f88351ca640 1 --2- 172.21.15.19:0/1426498120 >> v2:172.21.15.19:3300/0 conn(0x7f88300b3d60 0x7f88300af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7f88351ca640 1 -- 172.21.15.19:0/1426498120 >> 172.21.15.19:0/1426498120 conn(0x7f88300b6e00 msgr2=0x7f88300b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 10 librados: watch_flush enter 2023-10-06T13:53:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 10 librados: watch_flush exit 2023-10-06T13:53:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 20 client.4369.objecter shutdown clearing up homeless session... 2023-10-06T13:53:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 10 client.4369.objecter successfully canceled tick 2023-10-06T13:53:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 1 -- 172.21.15.19:0/694664086 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe06006c1c0 msgr2=0x7fe06006e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7f88351ca640 1 -- 172.21.15.19:0/1426498120 shutdown_connections 2023-10-06T13:53:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 1 --2- 172.21.15.19:0/694664086 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe06006c1c0 0x7fe06006e670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe0740ab760 tx=0x7fe068009040 comp rx=0 tx=0).stop 2023-10-06T13:53:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 10 monclient: shutdown 2023-10-06T13:53:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 1 -- 172.21.15.19:0/694664086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe05c0052c0 msgr2=0x7fe0740a9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 1 --2- 172.21.15.19:0/694664086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe05c0052c0 0x7fe0740a9f70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe06400b500 tx=0x7fe06400b9c0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 1 -- 172.21.15.19:0/694664086 shutdown_connections 2023-10-06T13:53:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 1 --2- 172.21.15.19:0/694664086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0740aca30 0x7fe0741d5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 1 --2- 172.21.15.19:0/694664086 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe06006c1c0 0x7fe06006e670 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 1 --2- 172.21.15.19:0/694664086 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe0740aaea0 0x7fe0740ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 1 --2- 172.21.15.19:0/694664086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe05c0052c0 0x7fe0740a9f70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 1 -- 172.21.15.19:0/694664086 >> 172.21.15.19:0/694664086 conn(0x7fe0740b6e00 msgr2=0x7fe0740b4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7f88351ca640 1 -- 172.21.15.19:0/1426498120 wait complete. 2023-10-06T13:53:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 1 -- 172.21.15.19:0/694664086 shutdown_connections 2023-10-06T13:53:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 1 -- 172.21.15.19:0/694664086 wait complete. 2023-10-06T13:53:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.462+0000 7fe07a0b6640 1 librados: shutdown 2023-10-06T13:53:22.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.463+0000 7f88351ca640 10 monclient: build_initial_monmap 2023-10-06T13:53:22.473 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.463+0000 7f88351ca640 10 monclient: monmap: 2023-10-06T13:53:22.473 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:22.473 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:22.473 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:22.474 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:22.474 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:22.474 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.474 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:22.474 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:22.474 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:22.474 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.464+0000 7f88351ca640 1 librados: starting msgr at 2023-10-06T13:53:22.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.464+0000 7f88351ca640 1 librados: starting objecter 2023-10-06T13:53:22.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.464+0000 7f88351ca640 1 Processor -- start 2023-10-06T13:53:22.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.464+0000 7f88351ca640 1 -- start start 2023-10-06T13:53:22.474 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.464+0000 7f88351ca640 1 librados: setting wanted keys 2023-10-06T13:53:22.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.464+0000 7f88351ca640 1 librados: calling monclient init 2023-10-06T13:53:22.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.464+0000 7f88351ca640 10 monclient: init 2023-10-06T13:53:22.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.464+0000 7f88351ca640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:22.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.464+0000 7f88351ca640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T13:53:22.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.464+0000 7f88351ca640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88300b00c0 0x7f88301d9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f88351ca640 10 monclient(hunting): picked mon.noname-a con 0x7f88300b00c0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:22.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f88351ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88300b3990 0x7f8830143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f88351ca640 10 monclient(hunting): picked mon.noname-b con 0x7f88300b3990 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:22.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f88351ca640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f88300b3d60 0x7f8830143bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f88351ca640 10 monclient(hunting): picked mon.noname-c con 0x7f88300b3d60 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:22.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f88351ca640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.475 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f88351ca640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8830144fd0 con 0x7f88300b00c0 2023-10-06T13:53:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f88351ca640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f88351ca640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8830143150 con 0x7f88300b3990 2023-10-06T13:53:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f88351ca640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f88351ca640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88300a8e70 con 0x7f88300b3d60 2023-10-06T13:53:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f88351ca640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f88351ca640 10 monclient(hunting): authenticate will time out at 1330.651669s 2023-10-06T13:53:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f882f59e640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f88300b3d60 0x7f8830143bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.465+0000 7f8827fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88300b3990 0x7f8830143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f882f59e640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f88300b3d60 0x7f8830143bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:41520/0 (socket says 172.21.15.19:41520) 2023-10-06T13:53:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f882f59e640 1 -- 172.21.15.19:0/793387049 learned_addr learned my addr 172.21.15.19:0/793387049 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f882f59e640 10 monclient(hunting): get_auth_request con 0x7f88300b3d60 auth_method 0 2023-10-06T13:53:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f882f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f882f59e640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f882f59e640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f8827fff640 10 monclient(hunting): get_auth_request con 0x7f88300b3990 auth_method 0 2023-10-06T13:53:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f8827fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f8827fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f8827fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f882f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f882f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f882f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f8827fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f8827fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.466+0000 7f8827fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.469+0000 7f8827fff640 10 monclient(hunting): handle_auth_done global_id 4375 payload 824 2023-10-06T13:53:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.469+0000 7f8827fff640 1 -- 172.21.15.19:0/793387049 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f88300b3d60 msgr2=0x7f8830143bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.469+0000 7f8827fff640 1 --2- 172.21.15.19:0/793387049 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f88300b3d60 0x7f8830143bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.469+0000 7f8827fff640 1 -- 172.21.15.19:0/793387049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88300b00c0 msgr2=0x7f88301d9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.469+0000 7f8827fff640 1 --2- 172.21.15.19:0/793387049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88300b00c0 0x7f88301d9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.469+0000 7f8827fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.469+0000 7f8827fff640 1 monclient: found mon.noname-b 2023-10-06T13:53:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.469+0000 7f8827fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.469+0000 7f8827fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.469+0000 7f8827fff640 1 -- 172.21.15.19:0/793387049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8828009b30 con 0x7f88300b3990 2023-10-06T13:53:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.471+0000 7f8827fff640 1 --2- 172.21.15.19:0/793387049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88300b3990 0x7f8830143670 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f881800b3e0 tx=0x7f881800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.471+0000 7f88277fe640 10 client.?.objecter ms_handle_connect 0x7f88300b3990 2023-10-06T13:53:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.471+0000 7f88277fe640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.473+0000 7f88277fe640 1 -- 172.21.15.19:0/793387049 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818011830 con 0x7f88300b3990 2023-10-06T13:53:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.473+0000 7f88277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.473+0000 7f88277fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.473+0000 7f88277fe640 10 monclient: dump: 2023-10-06T13:53:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.480 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.473+0000 7f88277fe640 10 monclient: _finish_auth 0 2023-10-06T13:53:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.473+0000 7f88277fe640 10 monclient: _check_auth_tickets 2023-10-06T13:53:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.473+0000 7f88277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.473+0000 7f88351ca640 5 monclient: authenticate success, global_id 4375 2023-10-06T13:53:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.473+0000 7f88351ca640 10 monclient: _renew_subs 2023-10-06T13:53:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.473+0000 7f88351ca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.473+0000 7f88351ca640 1 -- 172.21.15.19:0/793387049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88301da300 con 0x7f88300b3990 2023-10-06T13:53:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.474+0000 7f88351ca640 10 client.4375.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.474+0000 7f88351ca640 10 monclient: _renew_subs 2023-10-06T13:53:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.474+0000 7f88351ca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.474+0000 7f88351ca640 1 -- 172.21.15.19:0/793387049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88300aa440 con 0x7f88300b3990 2023-10-06T13:53:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.474+0000 7f88351ca640 1 librados: init done 2023-10-06T13:53:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.474+0000 7f88351ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:22.482 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.475+0000 7f88351ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:22.482 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.475+0000 7f88351ca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:22.482 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.475+0000 7f88351ca640 1 -- 172.21.15.19:0/793387049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88300678e0 con 0x7f88300b3990 2023-10-06T13:53:22.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2023-10-06T13:53:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.475+0000 7f88277fe640 1 -- 172.21.15.19:0/793387049 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8818009150 con 0x7f88300b3990 2023-10-06T13:53:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.475+0000 7f88277fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.475+0000 7f882dd9b640 4 set_mon_vals no callback set 2023-10-06T13:53:22.496 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.495+0000 7f2b343fd640 10 monclient: get_monmap_and_config 2023-10-06T13:53:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.495+0000 7f2b343fd640 10 monclient: build_initial_monmap 2023-10-06T13:53:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.495+0000 7f2b343fd640 10 monclient: monmap: 2023-10-06T13:53:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:22.496230+0000 2023-10-06T13:53:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:22.496230+0000 2023-10-06T13:53:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:22.498 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.495+0000 7f2b343fd640 1 Processor -- start 2023-10-06T13:53:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.495+0000 7f2b343fd640 1 -- start start 2023-10-06T13:53:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.495+0000 7f2b343fd640 10 monclient: init 2023-10-06T13:53:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.496+0000 7f2b343fd640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.496+0000 7f2b343fd640 10 monclient: _add_conns ranks=[0,1,4,2,3] 2023-10-06T13:53:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.496+0000 7f2b343fd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2b2c0b3990 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.496+0000 7f2b343fd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2b2c0b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.496+0000 7f2b343fd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2b2c0b07d0 0x7f2b2c0af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.496+0000 7f2b343fd640 10 monclient(hunting): picked mon.noname-c con 0x7f2b2c0b07d0 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:22.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.496+0000 7f2b343fd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2c0aebe0 con 0x7f2b2c0b3990 2023-10-06T13:53:22.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.496+0000 7f2b343fd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2c141070 con 0x7f2b2c0b3d60 2023-10-06T13:53:22.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.496+0000 7f2b343fd640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.496+0000 7f2b343fd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c140dd0 con 0x7f2b2c0b07d0 2023-10-06T13:53:22.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.496+0000 7f2b343fd640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:22.500 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.496+0000 7f2b343fd640 10 monclient(hunting): authenticate will time out at 1330.683156s 2023-10-06T13:53:22.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.546+0000 7f9cff7fe640 1 -- 172.21.15.19:0/4111966358 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f9d0c053af0 con 0x7f9d180ac020 2023-10-06T13:53:22.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.546+0000 7f9cff7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:22.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.546+0000 7f9cff7fe640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.547+0000 7fe043fff640 1 --2- 172.21.15.19:0/102371369 >> v2:172.21.15.19:3300/0 conn(0x7fe0440b07d0 0x7fe0440af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.551+0000 7fe043fff640 10 monclient(hunting): get_auth_request con 0x7fe0440b07d0 auth_method 0 2023-10-06T13:53:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.551+0000 7fe043fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.551+0000 7fe043fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.551+0000 7fe043fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:22.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.552+0000 7f2b32199640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2b2c0b3990 0x7f2b2c141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58024/0 (socket says 172.21.15.19:58024) 2023-10-06T13:53:22.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.552+0000 7f2b32199640 1 -- 172.21.15.19:0/3589808301 learned_addr learned my addr 172.21.15.19:0/3589808301 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:22.559 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.558+0000 7fe043fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:22.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.558+0000 7fe043fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:22.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.558+0000 7fe043fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:22.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.568+0000 7fe043fff640 10 monclient(hunting): handle_auth_done global_id 4347 payload 274 2023-10-06T13:53:22.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.568+0000 7fe043fff640 1 -- 172.21.15.19:0/102371369 >> v1:172.21.15.83:6789/0 conn(0x7fe0440b3d60 legacy=0x7fe0440b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.568+0000 7fe043fff640 1 -- 172.21.15.19:0/102371369 >> v1:172.21.15.19:6789/0 conn(0x7fe0440b3990 legacy=0x7fe044141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.568+0000 7fe043fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:22.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.568+0000 7fe043fff640 1 monclient: found mon.noname-a 2023-10-06T13:53:22.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.568+0000 7fe043fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:22.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.568+0000 7fe043fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:22.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.568+0000 7fe043fff640 1 -- 172.21.15.19:0/102371369 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe044141c10 con 0x7fe0440b07d0 2023-10-06T13:53:22.573 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.571+0000 7fe043fff640 1 --2- 172.21.15.19:0/102371369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0440b07d0 0x7fe0440af810 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009d90 tx=0x7fe03c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23544f9f5b1e087f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:22.659 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.657+0000 7f2b31197640 1 -- 172.21.15.19:0/3589808301 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973085993 0 0) 0x7f2b2c0aebe0 con 0x7f2b2c0b3990 2023-10-06T13:53:22.659 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.657+0000 7f2b31197640 10 monclient(hunting): _negotiate 2023-10-06T13:53:22.659 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.657+0000 7f2b31197640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:22.659 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.657+0000 7f2b31197640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:22.659 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.657+0000 7f2b31197640 10 monclient(hunting): my global_id is 4350 2023-10-06T13:53:22.660 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.657+0000 7f2b31197640 1 -- 172.21.15.19:0/3589808301 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b18003520 con 0x7f2b2c0b3990 2023-10-06T13:53:22.662 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.660+0000 7f48f7fff640 1 -- 172.21.15.19:0/4068037190 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4900044020 con 0x7f48f8003970 2023-10-06T13:53:22.662 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.660+0000 7f48f7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2023-10-06T13:53:22.662 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.660+0000 7f48f7fff640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:22.662 INFO:teuthology.orchestra.run.smithi019.stdout:60129542149 2023-10-06T13:53:22.665 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 10 librados: watch_flush enter 2023-10-06T13:53:22.665 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 10 librados: watch_flush exit 2023-10-06T13:53:22.665 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 20 client.4341.objecter shutdown clearing up homeless session... 2023-10-06T13:53:22.665 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 10 client.4341.objecter successfully canceled tick 2023-10-06T13:53:22.665 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 1 -- 172.21.15.19:0/4068037190 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f48fc06c210 msgr2=0x7f48fc06e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.665 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 1 --2- 172.21.15.19:0/4068037190 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f48fc06c210 0x7f48fc06e6c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4904000fd0 tx=0x7f4904001590 comp rx=0 tx=0).stop 2023-10-06T13:53:22.665 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 10 monclient: shutdown 2023-10-06T13:53:22.665 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:22.665 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 1 -- 172.21.15.19:0/4068037190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48f8003970 msgr2=0x7f4910144800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.665 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 1 --2- 172.21.15.19:0/4068037190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48f8003970 0x7f4910144800 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f490002bb10 tx=0x7f4900000fb0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 1 -- 172.21.15.19:0/4068037190 shutdown_connections 2023-10-06T13:53:22.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 1 --2- 172.21.15.19:0/4068037190 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f48fc06c210 0x7f48fc06e6c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 1 --2- 172.21.15.19:0/4068037190 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f49101395f0 0x7f4910139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 1 --2- 172.21.15.19:0/4068037190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f491013a260 0x7f49100acbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 1 --2- 172.21.15.19:0/4068037190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48f8003970 0x7f4910144800 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.662+0000 7f491589d640 1 -- 172.21.15.19:0/4068037190 >> 172.21.15.19:0/4068037190 conn(0x7f49100b6e00 msgr2=0x7f491013e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:22.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7f491589d640 1 -- 172.21.15.19:0/4068037190 shutdown_connections 2023-10-06T13:53:22.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7f491589d640 1 -- 172.21.15.19:0/4068037190 wait complete. 2023-10-06T13:53:22.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7f491589d640 1 librados: shutdown 2023-10-06T13:53:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe0427fc640 1 -- 172.21.15.19:0/102371369 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c02cc60 con 0x7fe0440b07d0 2023-10-06T13:53:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe0427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe0427fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe0427fc640 10 monclient: dump: 2023-10-06T13:53:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.669 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe0427fc640 10 monclient: _finish_auth 0 2023-10-06T13:53:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe0427fc640 10 monclient: _check_auth_tickets 2023-10-06T13:53:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe0427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe0427fc640 1 -- 172.21.15.19:0/102371369 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c02cdc0 con 0x7fe0440b07d0 2023-10-06T13:53:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe0427fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe0427fc640 1 -- 172.21.15.19:0/102371369 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c034660 con 0x7fe0440b07d0 2023-10-06T13:53:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe0427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe0427fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe0427fc640 10 monclient: dump: 2023-10-06T13:53:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.671 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe049ae2640 5 monclient: authenticate success, global_id 4347 2023-10-06T13:53:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe049ae2640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.663+0000 7fe049ae2640 4 set_mon_vals no callback set 2023-10-06T13:53:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.664+0000 7fe049ae2640 10 monclient: shutdown 2023-10-06T13:53:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.664+0000 7fe049ae2640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.664+0000 7fe049ae2640 1 -- 172.21.15.19:0/102371369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0440b07d0 msgr2=0x7fe0440af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.664+0000 7fe049ae2640 1 --2- 172.21.15.19:0/102371369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0440b07d0 0x7fe0440af810 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009d90 tx=0x7fe03c02beb0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.664+0000 7fe049ae2640 1 -- 172.21.15.19:0/102371369 shutdown_connections 2023-10-06T13:53:22.672 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.664+0000 7fe049ae2640 1 --2- 172.21.15.19:0/102371369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0440b07d0 0x7fe0440af810 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.672 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.664+0000 7fe049ae2640 1 -- 172.21.15.19:0/102371369 >> 172.21.15.19:0/102371369 conn(0x7fe0440b6e00 msgr2=0x7fe0440b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:22.672 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.664+0000 7fe049ae2640 1 -- 172.21.15.19:0/102371369 shutdown_connections 2023-10-06T13:53:22.672 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.664+0000 7fe049ae2640 1 -- 172.21.15.19:0/102371369 wait complete. 2023-10-06T13:53:22.672 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.665+0000 7fe049ae2640 10 monclient: build_initial_monmap 2023-10-06T13:53:22.672 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.665+0000 7fe049ae2640 10 monclient: monmap: 2023-10-06T13:53:22.672 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:22.672 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:22.674 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:22.674 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:22.674 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:22.676 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.665+0000 7fe049ae2640 1 librados: starting msgr at 2023-10-06T13:53:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.665+0000 7fe049ae2640 1 librados: starting objecter 2023-10-06T13:53:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.665+0000 7fe049ae2640 1 Processor -- start 2023-10-06T13:53:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.665+0000 7fe049ae2640 1 -- start start 2023-10-06T13:53:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.665+0000 7fe049ae2640 1 librados: setting wanted keys 2023-10-06T13:53:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.665+0000 7fe049ae2640 1 librados: calling monclient init 2023-10-06T13:53:22.677 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.665+0000 7fe049ae2640 10 monclient: init 2023-10-06T13:53:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 -- 172.21.15.19:0/3589808301 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1071371092 0 0) 0x7f2b18003520 con 0x7f2b2c0b3990 2023-10-06T13:53:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 -- 172.21.15.19:0/3589808301 >> v2:172.21.15.83:3300/0 conn(0x7f2b2c0b07d0 msgr2=0x7f2b2c0af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 --2- 172.21.15.19:0/3589808301 >> v2:172.21.15.83:3300/0 conn(0x7f2b2c0b07d0 0x7f2b2c0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 -- 172.21.15.19:0/3589808301 >> v1:172.21.15.83:6789/0 conn(0x7f2b2c0b3d60 legacy=0x7f2b2c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient: _finish_hunting 0 2023-10-06T13:53:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 monclient: found mon.noname-a-legacy 2023-10-06T13:53:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:22.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.19:6789/0 2023-10-06T13:53:22.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 -- 172.21.15.19:0/3589808301 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c141c10 con 0x7f2b2c0b3990 2023-10-06T13:53:22.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient: _finish_auth 0 2023-10-06T13:53:22.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient: _check_auth_tickets 2023-10-06T13:53:22.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:22.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 -- 172.21.15.19:0/3589808301 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251531364 0 0) 0x7f2b28001670 con 0x7f2b2c0b3990 2023-10-06T13:53:22.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b343fd640 5 monclient: authenticate success, global_id 4350 2023-10-06T13:53:22.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b343fd640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:22.682 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.682 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2023-10-06T13:53:22.682 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient: dump: 2023-10-06T13:53:22.682 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.682 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.682 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.682 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.683 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.683 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.683 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.683 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.683 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.683 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.683 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] but i'm connected to v1:172.21.15.19:6789/0, reconnecting 2023-10-06T13:53:22.684 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:22.684 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 -- 172.21.15.19:0/3589808301 >> v1:172.21.15.19:6789/0 conn(0x7f2b2c0b3990 legacy=0x7f2b2c141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.684 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:53:22.684 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 --2- 172.21.15.19:0/3589808301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b18003970 0x7f2b18003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.684 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient(hunting): picked mon.a con 0x7f2b18003970 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:22.684 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 --2- 172.21.15.19:0/3589808301 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2b180047c0 0x7f2b18004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.684 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient(hunting): picked mon.c con 0x7f2b180047c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 --2- 172.21.15.19:0/3589808301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b180052c0 0x7f2b1800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient(hunting): picked mon.b con 0x7f2b180052c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 -- 172.21.15.19:0/3589808301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c140dd0 con 0x7f2b18003970 2023-10-06T13:53:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 -- 172.21.15.19:0/3589808301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1800c1f0 con 0x7f2b180052c0 2023-10-06T13:53:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.686 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 1 -- 172.21.15.19:0/3589808301 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1800c3b0 con 0x7f2b180047c0 2023-10-06T13:53:22.686 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:22.686 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b31197640 10 monclient(hunting): _finish_auth 0 2023-10-06T13:53:22.686 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7f2b343fd640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:22.687 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:22.687 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 10 monclient: _add_conns ranks=[2,0,1] 2023-10-06T13:53:22.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe0440b07d0 0x7fe044136600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 10 monclient(hunting): picked mon.noname-c con 0x7fe0440b07d0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:22.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0440b3990 0x7fe044142f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 10 monclient(hunting): picked mon.noname-a con 0x7fe0440b3990 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:22.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0440b3d60 0x7fe044143490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 10 monclient(hunting): picked mon.noname-b con 0x7fe0440b3d60 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:22.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0441443f0 con 0x7fe0440b3990 2023-10-06T13:53:22.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe044144cf0 con 0x7fe0440b3d60 2023-10-06T13:53:22.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0441361b0 con 0x7fe0440b07d0 2023-10-06T13:53:22.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:22.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.666+0000 7fe049ae2640 10 monclient(hunting): authenticate will time out at 1330.853306s 2023-10-06T13:53:22.690 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.674+0000 7f2b32199640 1 --2- 172.21.15.19:0/3589808301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b18003970 0x7f2b18003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.690 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.674+0000 7f2b32199640 10 monclient(hunting): get_auth_request con 0x7f2b18003970 auth_method 0 2023-10-06T13:53:22.690 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.674+0000 7f2b32199640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:22.690 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.674+0000 7f2b32199640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:22.691 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.674+0000 7f2b32199640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:53:22.692 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.674+0000 7fe042ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0440b3990 0x7fe044142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.692 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.675+0000 7fe042ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0440b3990 0x7fe044142f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39106/0 (socket says 172.21.15.19:39106) 2023-10-06T13:53:22.692 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.675+0000 7fe042ffd640 1 -- 172.21.15.19:0/4280988172 learned_addr learned my addr 172.21.15.19:0/4280988172 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:22.692 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.675+0000 7fe042ffd640 10 monclient(hunting): get_auth_request con 0x7fe0440b3990 auth_method 0 2023-10-06T13:53:22.692 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.675+0000 7fe042ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:22.692 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.675+0000 7fe042ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:22.693 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.675+0000 7fe042ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:22.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2023-10-06T13:53:22.694 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.675+0000 7fe042ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:22.694 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.675+0000 7fe042ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:22.694 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.675+0000 7fe042ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.676+0000 7f2b32199640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.676+0000 7f2b32199640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.676+0000 7f2b32199640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T13:53:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b31998640 1 --2- 172.21.15.19:0/3589808301 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2b180047c0 0x7f2b18004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.696 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7fe0437fe640 1 --2- 172.21.15.19:0/4280988172 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe0440b07d0 0x7fe044136600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.697 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b31998640 10 monclient(hunting): get_auth_request con 0x7f2b180047c0 auth_method 0 2023-10-06T13:53:22.698 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b31998640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:22.698 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b31998640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:22.698 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b31998640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:53:22.698 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b32199640 10 monclient(hunting): handle_auth_done global_id 4350 payload 293 2023-10-06T13:53:22.698 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b32199640 1 -- 172.21.15.19:0/3589808301 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2b180047c0 msgr2=0x7f2b18004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:22.698 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b32199640 1 --2- 172.21.15.19:0/3589808301 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2b180047c0 0x7f2b18004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.699 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b32199640 1 -- 172.21.15.19:0/3589808301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b180052c0 msgr2=0x7f2b1800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:22.699 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b32199640 1 --2- 172.21.15.19:0/3589808301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b180052c0 0x7f2b1800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.699 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b32199640 10 monclient: _finish_hunting 0 2023-10-06T13:53:22.699 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b32199640 1 monclient: found mon.a 2023-10-06T13:53:22.699 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b32199640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:22.699 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b32199640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:22.699 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b3299a640 1 -- 172.21.15.19:0/3589808301 reap_dead start 2023-10-06T13:53:22.699 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b32199640 1 -- 172.21.15.19:0/3589808301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b18003520 con 0x7f2b18003970 2023-10-06T13:53:22.700 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b32199640 10 monclient: _finish_auth 0 2023-10-06T13:53:22.700 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b32199640 10 monclient: _check_auth_tickets 2023-10-06T13:53:22.700 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7f2b32199640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:22.700 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7fe0437fe640 10 monclient(hunting): get_auth_request con 0x7fe0440b07d0 auth_method 0 2023-10-06T13:53:22.701 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7fe0437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:22.701 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7fe0437fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:22.701 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.678+0000 7fe0437fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:22.701 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.680+0000 7fe0437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:22.701 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.680+0000 7fe0437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:22.701 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.680+0000 7fe0437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:22.702 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.680+0000 7f2b32199640 1 --2- 172.21.15.19:0/3589808301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b18003970 0x7f2b18003d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800c210 tx=0x7f2b180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ccb6b4fcac76bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:22.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.684+0000 7fe0437fe640 10 monclient(hunting): handle_auth_done global_id 4295 payload 824 2023-10-06T13:53:22.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.684+0000 7fe0437fe640 1 -- 172.21.15.19:0/4280988172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0440b3d60 msgr2=0x7fe044143490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.684+0000 7fe0437fe640 1 --2- 172.21.15.19:0/4280988172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0440b3d60 0x7fe044143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.684+0000 7fe0437fe640 1 -- 172.21.15.19:0/4280988172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0440b3990 msgr2=0x7fe044142f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.704 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.684+0000 7fe0437fe640 1 --2- 172.21.15.19:0/4280988172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0440b3990 0x7fe044142f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.704 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.684+0000 7fe0437fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:22.705 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.684+0000 7fe0437fe640 1 monclient: found mon.noname-c 2023-10-06T13:53:22.705 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.684+0000 7fe0437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:22.705 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.684+0000 7fe0437fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T13:53:22.705 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.684+0000 7fe0437fe640 1 -- 172.21.15.19:0/4280988172 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c009a70 con 0x7fe0440b07d0 2023-10-06T13:53:22.705 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.686+0000 7fe0437fe640 1 --2- 172.21.15.19:0/4280988172 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe0440b07d0 0x7fe044136600 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe03400c390 tx=0x7fe03400c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:22.705 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.686+0000 7fe02bfff640 10 client.?.objecter ms_handle_connect 0x7fe0440b07d0 2023-10-06T13:53:22.706 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.686+0000 7fe02bfff640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:22.706 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.689+0000 7f2b31197640 1 -- 172.21.15.19:0/3589808301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2802cae0 con 0x7f2b18003970 2023-10-06T13:53:22.706 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.689+0000 7f2b31197640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.706 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.689+0000 7f2b31197640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:22.707 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.689+0000 7f2b31197640 10 monclient: dump: 2023-10-06T13:53:22.707 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.707 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.707 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.707 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.707 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.707 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.707 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.707 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.707 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.707 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.707 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.689+0000 7f2b31197640 1 -- 172.21.15.19:0/3589808301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2802cc40 con 0x7f2b18003970 2023-10-06T13:53:22.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.689+0000 7f2b343fd640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:22.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.689+0000 7f2b31197640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:22.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.689+0000 7f2b343fd640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:22.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.689+0000 7f2b343fd640 4 set_mon_vals no callback set 2023-10-06T13:53:22.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.689+0000 7f2b31197640 1 -- 172.21.15.19:0/3589808301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b280058d0 con 0x7f2b18003970 2023-10-06T13:53:22.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.689+0000 7f2b31197640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.689+0000 7f2b31197640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:22.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.689+0000 7f2b31197640 10 monclient: dump: 2023-10-06T13:53:22.708 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.708 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.708 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.709 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.709 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.709 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.709 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.709 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.709 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.709 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.709 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe02bfff640 1 -- 172.21.15.19:0/4280988172 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0340052d0 con 0x7fe0440b07d0 2023-10-06T13:53:22.710 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe02bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.710 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe02bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:22.710 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe02bfff640 10 monclient: dump: 2023-10-06T13:53:22.710 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.710 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.710 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.710 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.710 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.710 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.710 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.710 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.710 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.711 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.711 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe02bfff640 10 monclient: _finish_auth 0 2023-10-06T13:53:22.711 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe02bfff640 10 monclient: _check_auth_tickets 2023-10-06T13:53:22.711 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe02bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:22.711 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7f2b343fd640 10 monclient: shutdown 2023-10-06T13:53:22.711 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7f2b343fd640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:22.711 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe02bfff640 1 -- 172.21.15.19:0/4280988172 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe034005430 con 0x7fe0440b07d0 2023-10-06T13:53:22.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe02bfff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:22.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe02bfff640 1 -- 172.21.15.19:0/4280988172 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034003830 con 0x7fe0440b07d0 2023-10-06T13:53:22.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe02bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe02bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:22.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7f2b343fd640 1 -- 172.21.15.19:0/3589808301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b18003970 msgr2=0x7f2b18003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.712 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7f2b343fd640 1 --2- 172.21.15.19:0/3589808301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b18003970 0x7f2b18003d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800c210 tx=0x7f2b180057f0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.713 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe02bfff640 10 monclient: dump: 2023-10-06T13:53:22.713 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.713 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.713 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.713 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.713 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.713 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.713 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.714 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.714 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.714 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.714 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe049ae2640 5 monclient: authenticate success, global_id 4295 2023-10-06T13:53:22.714 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe049ae2640 10 monclient: _renew_subs 2023-10-06T13:53:22.714 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe049ae2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:22.714 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe049ae2640 1 -- 172.21.15.19:0/4280988172 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0441445f0 con 0x7fe0440b07d0 2023-10-06T13:53:22.714 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe041ffb640 4 set_mon_vals no callback set 2023-10-06T13:53:22.714 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe049ae2640 10 client.4295.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:22.714 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe049ae2640 10 monclient: _renew_subs 2023-10-06T13:53:22.714 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe049ae2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:22.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe049ae2640 1 -- 172.21.15.19:0/4280988172 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe044131710 con 0x7fe0440b07d0 2023-10-06T13:53:22.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7fe049ae2640 1 librados: init done 2023-10-06T13:53:22.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7f2b343fd640 1 -- 172.21.15.19:0/3589808301 shutdown_connections 2023-10-06T13:53:22.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7f2b343fd640 1 --2- 172.21.15.19:0/3589808301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b18003970 0x7f2b18003d60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.690+0000 7f2b343fd640 1 -- 172.21.15.19:0/3589808301 >> 172.21.15.19:0/3589808301 conn(0x7f2b2c0b6e00 msgr2=0x7f2b2c0b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:22.716 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.691+0000 7f2b343fd640 1 -- 172.21.15.19:0/3589808301 shutdown_connections 2023-10-06T13:53:22.716 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.691+0000 7f2b343fd640 1 -- 172.21.15.19:0/3589808301 wait complete. 2023-10-06T13:53:22.716 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.691+0000 7fe049ae2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:22.716 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.691+0000 7fe049ae2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:22.716 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.691+0000 7fe049ae2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:22.716 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.691+0000 7fe049ae2640 1 -- 172.21.15.19:0/4280988172 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0440678e0 con 0x7fe0440b07d0 2023-10-06T13:53:22.719 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.692+0000 7f2b343fd640 10 monclient: build_initial_monmap 2023-10-06T13:53:22.719 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.692+0000 7f2b343fd640 10 monclient: monmap: 2023-10-06T13:53:22.719 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:22.719 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:22.719 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:22.719 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:22.719 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:22.719 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.719 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:22.720 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.692+0000 7f2b343fd640 1 librados: starting msgr at 2023-10-06T13:53:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.692+0000 7f2b343fd640 1 librados: starting objecter 2023-10-06T13:53:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.692+0000 7f2b343fd640 1 Processor -- start 2023-10-06T13:53:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.692+0000 7f2b343fd640 1 -- start start 2023-10-06T13:53:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.692+0000 7f2b343fd640 1 librados: setting wanted keys 2023-10-06T13:53:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.692+0000 7f2b343fd640 1 librados: calling monclient init 2023-10-06T13:53:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.692+0000 7f2b343fd640 10 monclient: init 2023-10-06T13:53:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:53:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b18003970 0x7f2b2c143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 10 monclient(hunting): picked mon.noname-a con 0x7f2b18003970 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2b2c0a9df0 0x7f2b2c13a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 10 monclient(hunting): picked mon.noname-c con 0x7f2b2c0a9df0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b2c13a720 0x7f2b2c1d5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 10 monclient(hunting): picked mon.noname-b con 0x7f2b2c13a720 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c144c80 con 0x7f2b18003970 2023-10-06T13:53:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0aa5e0 con 0x7f2b2c13a720 2023-10-06T13:53:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0aa240 con 0x7f2b2c0a9df0 2023-10-06T13:53:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.693+0000 7f2b343fd640 10 monclient(hunting): authenticate will time out at 1330.879909s 2023-10-06T13:53:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.703+0000 7f2b32199640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b18003970 0x7f2b2c143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.703+0000 7f2b32199640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b18003970 0x7f2b2c143f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39110/0 (socket says 172.21.15.19:39110) 2023-10-06T13:53:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.703+0000 7f2b32199640 1 -- 172.21.15.19:0/1253811082 learned_addr learned my addr 172.21.15.19:0/1253811082 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.703+0000 7f2b32199640 10 monclient(hunting): get_auth_request con 0x7f2b18003970 auth_method 0 2023-10-06T13:53:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.703+0000 7f2b32199640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.703+0000 7f2b32199640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.703+0000 7f2b32199640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.704+0000 7f2b32199640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.704+0000 7f2b32199640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.704+0000 7f2b32199640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.707+0000 7f2b32199640 10 monclient(hunting): handle_auth_done global_id 4356 payload 824 2023-10-06T13:53:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.707+0000 7f2b32199640 1 -- 172.21.15.19:0/1253811082 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2b2c0a9df0 msgr2=0x7f2b2c13a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.707+0000 7f2b32199640 1 --2- 172.21.15.19:0/1253811082 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2b2c0a9df0 0x7f2b2c13a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.707+0000 7f2b32199640 1 -- 172.21.15.19:0/1253811082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b2c13a720 msgr2=0x7f2b2c1d5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.707+0000 7f2b32199640 1 --2- 172.21.15.19:0/1253811082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b2c13a720 0x7f2b2c1d5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.707+0000 7f2b32199640 10 monclient: _finish_hunting 0 2023-10-06T13:53:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.707+0000 7f2b32199640 1 monclient: found mon.noname-a 2023-10-06T13:53:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.707+0000 7f2b32199640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.707+0000 7f2b32199640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.707+0000 7f2b32199640 1 -- 172.21.15.19:0/1253811082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c0aaba0 con 0x7f2b18003970 2023-10-06T13:53:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.709+0000 7f2b32199640 1 --2- 172.21.15.19:0/1253811082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b18003970 0x7f2b2c143f10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800c210 tx=0x7f2b28001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:22.724 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.709+0000 7f2b22ffd640 10 client.?.objecter ms_handle_connect 0x7f2b18003970 2023-10-06T13:53:22.724 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.709+0000 7f2b22ffd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:22.724 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.713+0000 7fe02bfff640 1 -- 172.21.15.19:0/4280988172 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe034003990 con 0x7fe0440b07d0 2023-10-06T13:53:22.724 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.713+0000 7fe02bfff640 1 --2- 172.21.15.19:0/4280988172 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe03006c1c0 0x7fe03006e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.713+0000 7f2b22ffd640 1 -- 172.21.15.19:0/1253811082 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28001630 con 0x7f2b18003970 2023-10-06T13:53:22.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.713+0000 7f2b22ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.713+0000 7f2b22ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:22.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.713+0000 7f2b22ffd640 10 monclient: dump: 2023-10-06T13:53:22.725 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.725 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.725 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.725 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.726 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.726 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.726 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.726 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.726 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.726 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.726 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b22ffd640 10 monclient: _finish_auth 0 2023-10-06T13:53:22.726 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b22ffd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:22.726 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b22ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:22.726 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b343fd640 5 monclient: authenticate success, global_id 4356 2023-10-06T13:53:22.726 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b22ffd640 1 -- 172.21.15.19:0/1253811082 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2803a070 con 0x7f2b18003970 2023-10-06T13:53:22.726 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b343fd640 10 monclient: _renew_subs 2023-10-06T13:53:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b343fd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b343fd640 1 -- 172.21.15.19:0/1253811082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2c1d5f80 con 0x7f2b18003970 2023-10-06T13:53:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b22ffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b30996640 4 set_mon_vals no callback set 2023-10-06T13:53:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7fe042ffd640 1 --2- 172.21.15.19:0/4280988172 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe03006c1c0 0x7fe03006e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7fe042ffd640 10 monclient: get_auth_request con 0x7fe03006c1c0 auth_method 0 2023-10-06T13:53:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b343fd640 10 client.4356.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b343fd640 10 monclient: _renew_subs 2023-10-06T13:53:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b343fd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b343fd640 1 -- 172.21.15.19:0/1253811082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c1d64e0 con 0x7f2b18003970 2023-10-06T13:53:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b343fd640 1 librados: init done 2023-10-06T13:53:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b22ffd640 1 -- 172.21.15.19:0/1253811082 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28001630 con 0x7f2b18003970 2023-10-06T13:53:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b22ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b22ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7f2b22ffd640 10 monclient: dump: 2023-10-06T13:53:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.730 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7fe042ffd640 1 --2- 172.21.15.19:0/4280988172 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe03006c1c0 0x7fe03006e670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe044142500 tx=0x7fe038009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:22.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.714+0000 7fe02bfff640 10 client.4295.objecter ms_handle_connect 0x7fe03006c1c0 2023-10-06T13:53:22.730 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.715+0000 7f2b343fd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:22.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.715+0000 7f2b343fd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:22.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.715+0000 7f2b343fd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:22.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.715+0000 7f2b343fd640 1 -- 172.21.15.19:0/1253811082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2c0aa7a0 con 0x7f2b18003970 2023-10-06T13:53:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.716+0000 7f88277fe640 1 -- 172.21.15.19:0/793387049 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818011b90 con 0x7f88300b3990 2023-10-06T13:53:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.716+0000 7f88277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.716+0000 7f88277fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.716+0000 7f88277fe640 10 monclient: dump: 2023-10-06T13:53:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:22.733 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.716+0000 7f88277fe640 1 -- 172.21.15.19:0/793387049 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f881801a400 con 0x7f88300b3990 2023-10-06T13:53:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.717+0000 7f88277fe640 1 --2- 172.21.15.19:0/793387049 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f881006c1c0 0x7f881006e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.717+0000 7f882ed9d640 1 --2- 172.21.15.19:0/793387049 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f881006c1c0 0x7f881006e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.717+0000 7f882ed9d640 10 monclient: get_auth_request con 0x7f881006c1c0 auth_method 0 2023-10-06T13:53:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.717+0000 7f88277fe640 10 client.4375.objecter ms_handle_connect 0x7f881006c1c0 2023-10-06T13:53:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.717+0000 7f882ed9d640 1 --2- 172.21.15.19:0/793387049 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f881006c1c0 0x7f881006e670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8820000f60 tx=0x7f88200079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.720+0000 7f88277fe640 1 -- 172.21.15.19:0/793387049 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8818058c30 con 0x7f88300b3990 2023-10-06T13:53:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.720+0000 7f88277fe640 10 client.4375.objecter ms_dispatch 0x7f8830000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.720+0000 7f88277fe640 3 client.4375.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:22.734 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.720+0000 7f88277fe640 3 client.4375.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:22.734 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.720+0000 7f88277fe640 20 client.4375.objecter dump_active .. 0 homeless 2023-10-06T13:53:22.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.736+0000 7f88277fe640 1 -- 172.21.15.19:0/793387049 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f8818058300 con 0x7f88300b3990 2023-10-06T13:53:22.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.736+0000 7f88277fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:22.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.736+0000 7f88277fe640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:22.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.738+0000 7f2b22ffd640 1 -- 172.21.15.19:0/1253811082 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f2b28032070 con 0x7f2b18003970 2023-10-06T13:53:22.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.739+0000 7f2b22ffd640 1 --2- 172.21.15.19:0/1253811082 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f2b1006c250 0x7f2b1006e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:22.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.739+0000 7f2b31998640 1 --2- 172.21.15.19:0/1253811082 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f2b1006c250 0x7f2b1006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:22.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.739+0000 7f2b31998640 10 monclient: get_auth_request con 0x7f2b1006c250 auth_method 0 2023-10-06T13:53:22.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.740+0000 7f2b22ffd640 10 client.4356.objecter ms_handle_connect 0x7f2b1006c250 2023-10-06T13:53:22.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.740+0000 7f2b31998640 1 --2- 172.21.15.19:0/1253811082 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f2b1006c250 0x7f2b1006e700 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0abd80 tx=0x7f2b1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.745+0000 7f2b22ffd640 1 -- 172.21.15.19:0/1253811082 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2b280797b0 con 0x7f2b18003970 2023-10-06T13:53:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.745+0000 7f2b22ffd640 10 client.4356.objecter ms_dispatch 0x7f2b2c000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.745+0000 7f2b22ffd640 3 client.4356.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.745+0000 7f2b22ffd640 3 client.4356.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.745+0000 7f2b22ffd640 20 client.4356.objecter dump_active .. 0 homeless 2023-10-06T13:53:22.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.746+0000 7f9d1d246640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2023-10-06T13:53:22.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.746+0000 7f9d1d246640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-10-06T13:53:22.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.746+0000 7f9d1d246640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:22.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.746+0000 7f9d1d246640 1 -- 172.21.15.19:0/4111966358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9d181d9300 con 0x7f9d180ac020 2023-10-06T13:53:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.919+0000 7f88351ca640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2023-10-06T13:53:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.919+0000 7f88351ca640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2023-10-06T13:53:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.919+0000 7f88351ca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.919+0000 7f88351ca640 1 -- 172.21.15.19:0/793387049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f88300ae200 con 0x7f88300b3990 2023-10-06T13:53:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.923+0000 7f88277fe640 1 -- 172.21.15.19:0/793387049 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8818057e90 con 0x7f88300b3990 2023-10-06T13:53:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.923+0000 7f88277fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2023-10-06T13:53:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.923+0000 7f88277fe640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:22.925 INFO:teuthology.orchestra.run.smithi019.stdout:68719476741 2023-10-06T13:53:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 10 librados: watch_flush enter 2023-10-06T13:53:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 10 librados: watch_flush exit 2023-10-06T13:53:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 20 client.4375.objecter shutdown clearing up homeless session... 2023-10-06T13:53:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 10 client.4375.objecter successfully canceled tick 2023-10-06T13:53:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 1 -- 172.21.15.19:0/793387049 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f881006c1c0 msgr2=0x7f881006e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 1 --2- 172.21.15.19:0/793387049 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f881006c1c0 0x7f881006e670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8820000f60 tx=0x7f88200079b0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 10 monclient: shutdown 2023-10-06T13:53:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 1 -- 172.21.15.19:0/793387049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88300b3990 msgr2=0x7f8830143670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 1 --2- 172.21.15.19:0/793387049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88300b3990 0x7f8830143670 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f881800b3e0 tx=0x7f881800b8a0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 1 -- 172.21.15.19:0/793387049 shutdown_connections 2023-10-06T13:53:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 1 --2- 172.21.15.19:0/793387049 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f881006c1c0 0x7f881006e670 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 1 --2- 172.21.15.19:0/793387049 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f88300b3d60 0x7f8830143bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 1 --2- 172.21.15.19:0/793387049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88300b3990 0x7f8830143670 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 1 --2- 172.21.15.19:0/793387049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88300b00c0 0x7f88301d9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 1 -- 172.21.15.19:0/793387049 >> 172.21.15.19:0/793387049 conn(0x7f88300b6e00 msgr2=0x7f883013cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 1 -- 172.21.15.19:0/793387049 shutdown_connections 2023-10-06T13:53:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 1 -- 172.21.15.19:0/793387049 wait complete. 2023-10-06T13:53:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.925+0000 7f88351ca640 1 librados: shutdown 2023-10-06T13:53:22.940 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476741 for osd.0 2023-10-06T13:53:22.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.977+0000 7fe02bfff640 1 -- 172.21.15.19:0/4280988172 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe034057930 con 0x7fe0440b07d0 2023-10-06T13:53:22.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.977+0000 7fe02bfff640 10 client.4295.objecter ms_dispatch 0x7fe044000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.977+0000 7fe02bfff640 3 client.4295.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.977+0000 7fe02bfff640 3 client.4295.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.977+0000 7fe02bfff640 20 client.4295.objecter dump_active .. 0 homeless 2023-10-06T13:53:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.977+0000 7fe02bfff640 1 -- 172.21.15.19:0/4280988172 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7fe034053c70 con 0x7fe0440b07d0 2023-10-06T13:53:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.977+0000 7fe02bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:22.977+0000 7fe02bfff640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:23.002 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.000+0000 7f9cff7fe640 1 -- 172.21.15.19:0/4111966358 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9d0c018030 con 0x7f9d180ac020 2023-10-06T13:53:23.002 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.000+0000 7f9cff7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-10-06T13:53:23.002 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.000+0000 7f9cff7fe640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:23.002 INFO:teuthology.orchestra.run.smithi019.stdout:64424509445 2023-10-06T13:53:23.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 10 librados: watch_flush enter 2023-10-06T13:53:23.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 10 librados: watch_flush exit 2023-10-06T13:53:23.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 20 client.4344.objecter shutdown clearing up homeless session... 2023-10-06T13:53:23.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 10 client.4344.objecter successfully canceled tick 2023-10-06T13:53:23.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 1 -- 172.21.15.19:0/4111966358 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9d0406c1c0 msgr2=0x7f9d0406e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 1 --2- 172.21.15.19:0/4111966358 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9d0406c1c0 0x7f9d0406e670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9d08035000 tx=0x7f9d0802d070 comp rx=0 tx=0).stop 2023-10-06T13:53:23.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 10 monclient: shutdown 2023-10-06T13:53:23.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:23.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 1 -- 172.21.15.19:0/4111966358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d180ac020 msgr2=0x7f9d181d9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 1 --2- 172.21.15.19:0/4111966358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d180ac020 0x7f9d181d9b90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00c710 tx=0x7f9d0c00cbd0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 1 -- 172.21.15.19:0/4111966358 shutdown_connections 2023-10-06T13:53:23.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 1 --2- 172.21.15.19:0/4111966358 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9d0406c1c0 0x7f9d0406e670 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 1 --2- 172.21.15.19:0/4111966358 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9d180a9ef0 0x7f9d180aa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 1 --2- 172.21.15.19:0/4111966358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d180ac020 0x7f9d181d9b90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 1 --2- 172.21.15.19:0/4111966358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d00003970 0x7f9d181d9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 1 -- 172.21.15.19:0/4111966358 >> 172.21.15.19:0/4111966358 conn(0x7f9d180b6e00 msgr2=0x7f9d180b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:23.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 1 -- 172.21.15.19:0/4111966358 shutdown_connections 2023-10-06T13:53:23.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 1 -- 172.21.15.19:0/4111966358 wait complete. 2023-10-06T13:53:23.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.002+0000 7f9d1d246640 1 librados: shutdown 2023-10-06T13:53:23.015 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2023-10-06T13:53:23.026 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.024+0000 7f2b22ffd640 1 -- 172.21.15.19:0/1253811082 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f2b28075af0 con 0x7f2b18003970 2023-10-06T13:53:23.026 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.024+0000 7f2b22ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:23.026 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.024+0000 7f2b22ffd640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:23.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.170+0000 7fe049ae2640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2023-10-06T13:53:23.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.170+0000 7fe049ae2640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2023-10-06T13:53:23.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.170+0000 7fe049ae2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:23.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.170+0000 7fe049ae2640 1 -- 172.21.15.19:0/4280988172 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe044144e30 con 0x7fe0440b07d0 2023-10-06T13:53:23.175 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.174+0000 7fe02bfff640 1 -- 172.21.15.19:0/4280988172 <== mon.2 v2:172.21.15.19:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe034017020 con 0x7fe0440b07d0 2023-10-06T13:53:23.175 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.174+0000 7fe02bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2023-10-06T13:53:23.175 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.174+0000 7fe02bfff640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:23.176 INFO:teuthology.orchestra.run.smithi019.stdout:55834574853 2023-10-06T13:53:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.175+0000 7fe049ae2640 10 librados: watch_flush enter 2023-10-06T13:53:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.175+0000 7fe049ae2640 10 librados: watch_flush exit 2023-10-06T13:53:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.175+0000 7fe049ae2640 20 client.4295.objecter shutdown clearing up homeless session... 2023-10-06T13:53:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.175+0000 7fe049ae2640 10 client.4295.objecter successfully canceled tick 2023-10-06T13:53:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 1 -- 172.21.15.19:0/4280988172 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe03006c1c0 msgr2=0x7fe03006e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 1 --2- 172.21.15.19:0/4280988172 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe03006c1c0 0x7fe03006e670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe044142500 tx=0x7fe038009040 comp rx=0 tx=0).stop 2023-10-06T13:53:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 10 monclient: shutdown 2023-10-06T13:53:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 1 -- 172.21.15.19:0/4280988172 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe0440b07d0 msgr2=0x7fe044136600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 1 --2- 172.21.15.19:0/4280988172 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe0440b07d0 0x7fe044136600 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe03400c390 tx=0x7fe03400c850 comp rx=0 tx=0).stop 2023-10-06T13:53:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 1 -- 172.21.15.19:0/4280988172 shutdown_connections 2023-10-06T13:53:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 1 --2- 172.21.15.19:0/4280988172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0440b3d60 0x7fe044143490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 1 --2- 172.21.15.19:0/4280988172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0440b3990 0x7fe044142f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 1 --2- 172.21.15.19:0/4280988172 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe03006c1c0 0x7fe03006e670 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 1 --2- 172.21.15.19:0/4280988172 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe0440b07d0 0x7fe044136600 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 1 -- 172.21.15.19:0/4280988172 >> 172.21.15.19:0/4280988172 conn(0x7fe0440b6e00 msgr2=0x7fe044139710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 1 -- 172.21.15.19:0/4280988172 shutdown_connections 2023-10-06T13:53:23.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 1 -- 172.21.15.19:0/4280988172 wait complete. 2023-10-06T13:53:23.180 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.176+0000 7fe049ae2640 1 librados: shutdown 2023-10-06T13:53:23.189 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.7 2023-10-06T13:53:23.190 DEBUG:teuthology.parallel:result is None 2023-10-06T13:53:23.210 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T13:53:23.213 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.211+0000 7f2b343fd640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2023-10-06T13:53:23.213 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.211+0000 7f2b343fd640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2023-10-06T13:53:23.213 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.211+0000 7f2b343fd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:23.213 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.211+0000 7f2b343fd640 1 -- 172.21.15.19:0/1253811082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2b2c0aad30 con 0x7f2b18003970 2023-10-06T13:53:23.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.216+0000 7f2b22ffd640 1 -- 172.21.15.19:0/1253811082 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2b2804e020 con 0x7f2b18003970 2023-10-06T13:53:23.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.216+0000 7f2b22ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2023-10-06T13:53:23.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.216+0000 7f2b22ffd640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:23.219 INFO:teuthology.orchestra.run.smithi019.stdout:64424509447 2023-10-06T13:53:23.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.218+0000 7f2b343fd640 10 librados: watch_flush enter 2023-10-06T13:53:23.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 10 librados: watch_flush exit 2023-10-06T13:53:23.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 20 client.4356.objecter shutdown clearing up homeless session... 2023-10-06T13:53:23.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 10 client.4356.objecter successfully canceled tick 2023-10-06T13:53:23.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 1 -- 172.21.15.19:0/1253811082 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f2b1006c250 msgr2=0x7f2b1006e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 1 --2- 172.21.15.19:0/1253811082 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f2b1006c250 0x7f2b1006e700 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0abd80 tx=0x7f2b1c009040 comp rx=0 tx=0).stop 2023-10-06T13:53:23.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 10 monclient: shutdown 2023-10-06T13:53:23.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:23.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 1 -- 172.21.15.19:0/1253811082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b18003970 msgr2=0x7f2b2c143f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 1 --2- 172.21.15.19:0/1253811082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b18003970 0x7f2b2c143f10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800c210 tx=0x7f2b28001a00 comp rx=0 tx=0).stop 2023-10-06T13:53:23.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 1 -- 172.21.15.19:0/1253811082 shutdown_connections 2023-10-06T13:53:23.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 1 --2- 172.21.15.19:0/1253811082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b2c13a720 0x7f2b2c1d5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 1 --2- 172.21.15.19:0/1253811082 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f2b1006c250 0x7f2b1006e700 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 1 --2- 172.21.15.19:0/1253811082 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2b2c0a9df0 0x7f2b2c13a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 1 --2- 172.21.15.19:0/1253811082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b18003970 0x7f2b2c143f10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 1 -- 172.21.15.19:0/1253811082 >> 172.21.15.19:0/1253811082 conn(0x7f2b2c0b6e00 msgr2=0x7f2b2c13e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:23.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 1 -- 172.21.15.19:0/1253811082 shutdown_connections 2023-10-06T13:53:23.223 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 1 -- 172.21.15.19:0/1253811082 wait complete. 2023-10-06T13:53:23.223 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.219+0000 7f2b343fd640 1 librados: shutdown 2023-10-06T13:53:23.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.3 2023-10-06T13:53:23.232 DEBUG:teuthology.parallel:result is None 2023-10-06T13:53:23.254 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T13:53:23.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.321+0000 7feabef0a640 10 monclient: get_monmap_and_config 2023-10-06T13:53:23.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.321+0000 7feabef0a640 10 monclient: build_initial_monmap 2023-10-06T13:53:23.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.321+0000 7feabef0a640 10 monclient: monmap: 2023-10-06T13:53:23.324 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:23.324 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.325 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:23.322684+0000 2023-10-06T13:53:23.325 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:23.322684+0000 2023-10-06T13:53:23.325 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:23.325 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.325 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:23.325 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:23.326 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:23.326 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:23.326 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:23.326 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.328 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.322+0000 7feabef0a640 1 Processor -- start 2023-10-06T13:53:23.328 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.322+0000 7feabef0a640 1 -- start start 2023-10-06T13:53:23.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.322+0000 7feabef0a640 10 monclient: init 2023-10-06T13:53:23.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabef0a640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:23.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabef0a640 10 monclient: _add_conns ranks=[1,3,4,2,0] 2023-10-06T13:53:23.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabef0a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7feab80b3990 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:23.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabef0a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feab80b3d60 0x7feab80af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabef0a640 10 monclient(hunting): picked mon.noname-a con 0x7feab80b3d60 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:23.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabef0a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feab80b00c0 0x7feab80b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabef0a640 10 monclient(hunting): picked mon.noname-c con 0x7feab80b00c0 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:23.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabef0a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab80aebe0 con 0x7feab80b3990 2023-10-06T13:53:23.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabef0a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabef0a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab80b0a60 con 0x7feab80b3d60 2023-10-06T13:53:23.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabef0a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabef0a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab8140dd0 con 0x7feab80b00c0 2023-10-06T13:53:23.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabef0a640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:23.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabef0a640 10 monclient(hunting): authenticate will time out at 1331.509486s 2023-10-06T13:53:23.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feab7fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feab80b3d60 0x7feab80af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feabd4a7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feab80b00c0 0x7feab80b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.323+0000 7feab7fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feab80b3d60 0x7feab80af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39122/0 (socket says 172.21.15.19:39122) 2023-10-06T13:53:23.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.324+0000 7feab7fff640 1 -- 172.21.15.19:0/556727663 learned_addr learned my addr 172.21.15.19:0/556727663 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:23.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.324+0000 7feab7fff640 10 monclient(hunting): get_auth_request con 0x7feab80b3d60 auth_method 0 2023-10-06T13:53:23.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.324+0000 7feab7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:23.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.324+0000 7feab7fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:23.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.324+0000 7feab7fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:23.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.324+0000 7feabd4a7640 10 monclient(hunting): get_auth_request con 0x7feab80b00c0 auth_method 0 2023-10-06T13:53:23.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.324+0000 7feabd4a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:23.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.324+0000 7feabd4a7640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:23.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.324+0000 7feabd4a7640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:23.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.324+0000 7feab7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:23.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.324+0000 7feab7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:23.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.324+0000 7feab7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:23.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.325+0000 7feabd4a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:23.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.325+0000 7feabd4a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:23.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.325+0000 7feabd4a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:23.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.326+0000 7feab7fff640 10 monclient(hunting): handle_auth_done global_id 4359 payload 274 2023-10-06T13:53:23.331 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.326+0000 7feab7fff640 1 -- 172.21.15.19:0/556727663 >> v2:172.21.15.83:3300/0 conn(0x7feab80b00c0 msgr2=0x7feab80b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:23.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.326+0000 7feab7fff640 1 --2- 172.21.15.19:0/556727663 >> v2:172.21.15.83:3300/0 conn(0x7feab80b00c0 0x7feab80b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.326+0000 7feab7fff640 1 -- 172.21.15.19:0/556727663 >> v1:172.21.15.83:6789/0 conn(0x7feab80b3990 legacy=0x7feab8141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.326+0000 7feab7fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:23.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.326+0000 7feab7fff640 1 monclient: found mon.noname-a 2023-10-06T13:53:23.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.326+0000 7feab7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:23.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.326+0000 7feab7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:23.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.326+0000 7feab7fff640 1 -- 172.21.15.19:0/556727663 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab8141070 con 0x7feab80b3d60 2023-10-06T13:53:23.332 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.328+0000 7feab7fff640 1 --2- 172.21.15.19:0/556727663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feab80b3d60 0x7feab80af810 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feab80ae820 tx=0x7feaa402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9369521001724b60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:23.333 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feab77fe640 1 -- 172.21.15.19:0/556727663 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4031870 con 0x7feab80b3d60 2023-10-06T13:53:23.334 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feab77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.334 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feab77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:23.334 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feab77fe640 10 monclient: dump: 2023-10-06T13:53:23.334 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.334 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.334 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.334 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.334 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.334 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.334 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.335 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.335 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.335 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feab77fe640 10 monclient: _finish_auth 0 2023-10-06T13:53:23.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feab77fe640 10 monclient: _check_auth_tickets 2023-10-06T13:53:23.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feab77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:23.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feab77fe640 1 -- 172.21.15.19:0/556727663 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa40319d0 con 0x7feab80b3d60 2023-10-06T13:53:23.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feabef0a640 5 monclient: authenticate success, global_id 4359 2023-10-06T13:53:23.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feabef0a640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:23.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feab77fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:23.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feab77fe640 1 -- 172.21.15.19:0/556727663 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4031d60 con 0x7feab80b3d60 2023-10-06T13:53:23.335 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feab77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feab77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:23.336 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feab77fe640 10 monclient: dump: 2023-10-06T13:53:23.336 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.336 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.336 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.336 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.336 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.336 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.336 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.336 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.336 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.336 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.337 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feabef0a640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:23.337 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.332+0000 7feabef0a640 4 set_mon_vals no callback set 2023-10-06T13:53:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.333+0000 7feabef0a640 10 monclient: shutdown 2023-10-06T13:53:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.333+0000 7feabef0a640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.333+0000 7feabef0a640 1 -- 172.21.15.19:0/556727663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feab80b3d60 msgr2=0x7feab80af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.333+0000 7feabef0a640 1 --2- 172.21.15.19:0/556727663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feab80b3d60 0x7feab80af810 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feab80ae820 tx=0x7feaa402b910 comp rx=0 tx=0).stop 2023-10-06T13:53:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.333+0000 7feabef0a640 1 -- 172.21.15.19:0/556727663 shutdown_connections 2023-10-06T13:53:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.333+0000 7feabef0a640 1 --2- 172.21.15.19:0/556727663 >> v2:172.21.15.83:3300/0 conn(0x7feab80b00c0 0x7feab80b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.333+0000 7feabef0a640 1 --2- 172.21.15.19:0/556727663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feab80b3d60 0x7feab80af810 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.333+0000 7feabef0a640 1 -- 172.21.15.19:0/556727663 >> 172.21.15.19:0/556727663 conn(0x7feab80b6e00 msgr2=0x7feab80b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.333+0000 7feabef0a640 1 -- 172.21.15.19:0/556727663 shutdown_connections 2023-10-06T13:53:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.333+0000 7feabef0a640 1 -- 172.21.15.19:0/556727663 wait complete. 2023-10-06T13:53:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.334+0000 7feabef0a640 10 monclient: build_initial_monmap 2023-10-06T13:53:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.334+0000 7feabef0a640 10 monclient: monmap: 2023-10-06T13:53:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:23.342 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.334+0000 7feabef0a640 1 librados: starting msgr at 2023-10-06T13:53:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.334+0000 7feabef0a640 1 librados: starting objecter 2023-10-06T13:53:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.334+0000 7feabef0a640 1 Processor -- start 2023-10-06T13:53:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.334+0000 7feabef0a640 1 -- start start 2023-10-06T13:53:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.334+0000 7feabef0a640 1 librados: setting wanted keys 2023-10-06T13:53:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.334+0000 7feabef0a640 1 librados: calling monclient init 2023-10-06T13:53:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.334+0000 7feabef0a640 10 monclient: init 2023-10-06T13:53:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T13:53:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab80b00c0 0x7feab80ac300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 10 monclient(hunting): picked mon.noname-b con 0x7feab80b00c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7feab80b3990 0x7feab81d5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 10 monclient(hunting): picked mon.noname-c con 0x7feab80b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feab80b3d60 0x7feab81d5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 10 monclient(hunting): picked mon.noname-a con 0x7feab80b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab81ca840 con 0x7feab80b3d60 2023-10-06T13:53:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab81c9090 con 0x7feab80b00c0 2023-10-06T13:53:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab80ace50 con 0x7feab80b3990 2023-10-06T13:53:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.335+0000 7feabef0a640 10 monclient(hunting): authenticate will time out at 1331.522312s 2023-10-06T13:53:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feabd4a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feab80b3d60 0x7feab81d5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feabcca6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab80b00c0 0x7feab80ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feab7fff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7feab80b3990 0x7feab81d5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feabd4a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feab80b3d60 0x7feab81d5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39126/0 (socket says 172.21.15.19:39126) 2023-10-06T13:53:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feabd4a7640 1 -- 172.21.15.19:0/4003648979 learned_addr learned my addr 172.21.15.19:0/4003648979 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feabd4a7640 10 monclient(hunting): get_auth_request con 0x7feab80b3d60 auth_method 0 2023-10-06T13:53:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feabd4a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feabd4a7640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feabd4a7640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feabcca6640 10 monclient(hunting): get_auth_request con 0x7feab80b00c0 auth_method 0 2023-10-06T13:53:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feabcca6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feabcca6640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feabcca6640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feab7fff640 10 monclient(hunting): get_auth_request con 0x7feab80b3990 auth_method 0 2023-10-06T13:53:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feab7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feab7fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.336+0000 7feab7fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.337+0000 7feabcca6640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.337+0000 7feabcca6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.337+0000 7feabcca6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.337+0000 7feabd4a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.337+0000 7feabd4a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.337+0000 7feabd4a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.337+0000 7feab7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.337+0000 7feab7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.337+0000 7feab7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.339+0000 7feabcca6640 10 monclient(hunting): handle_auth_done global_id 4384 payload 824 2023-10-06T13:53:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.340+0000 7feabcca6640 1 -- 172.21.15.19:0/4003648979 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7feab80b3990 msgr2=0x7feab81d5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.340+0000 7feabcca6640 1 --2- 172.21.15.19:0/4003648979 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7feab80b3990 0x7feab81d5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.340+0000 7feabcca6640 1 -- 172.21.15.19:0/4003648979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feab80b3d60 msgr2=0x7feab81d5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.340+0000 7feabcca6640 1 --2- 172.21.15.19:0/4003648979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feab80b3d60 0x7feab81d5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.340+0000 7feabcca6640 10 monclient: _finish_hunting 0 2023-10-06T13:53:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.340+0000 7feabcca6640 1 monclient: found mon.noname-b 2023-10-06T13:53:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.340+0000 7feabcca6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.340+0000 7feabcca6640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.340+0000 7feabcca6640 1 -- 172.21.15.19:0/4003648979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa40094d0 con 0x7feab80b00c0 2023-10-06T13:53:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.342+0000 7feabcca6640 1 --2- 172.21.15.19:0/4003648979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab80b00c0 0x7feab80ac300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feaac00c390 tx=0x7feaac00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.342+0000 7feab57fa640 10 client.?.objecter ms_handle_connect 0x7feab80b00c0 2023-10-06T13:53:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.342+0000 7feab57fa640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feab57fa640 1 -- 172.21.15.19:0/4003648979 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac0052d0 con 0x7feab80b00c0 2023-10-06T13:53:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feab57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feab57fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feab57fa640 10 monclient: dump: 2023-10-06T13:53:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.350 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feab57fa640 10 monclient: _finish_auth 0 2023-10-06T13:53:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feab57fa640 10 monclient: _check_auth_tickets 2023-10-06T13:53:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feab57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feab57fa640 1 -- 172.21.15.19:0/4003648979 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaac005430 con 0x7feab80b00c0 2023-10-06T13:53:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feab57fa640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feab57fa640 1 -- 172.21.15.19:0/4003648979 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac003830 con 0x7feab80b00c0 2023-10-06T13:53:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feabef0a640 5 monclient: authenticate success, global_id 4384 2023-10-06T13:53:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feabef0a640 10 monclient: _renew_subs 2023-10-06T13:53:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feabef0a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feab6ffd640 4 set_mon_vals no callback set 2023-10-06T13:53:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feabef0a640 1 -- 172.21.15.19:0/4003648979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab81c8cb0 con 0x7feab80b00c0 2023-10-06T13:53:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feab57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feab57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feab57fa640 10 monclient: dump: 2023-10-06T13:53:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.352 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.352 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.352 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.352 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feabef0a640 10 client.4384.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:23.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feabef0a640 10 monclient: _renew_subs 2023-10-06T13:53:23.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feabef0a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:23.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feabef0a640 1 -- 172.21.15.19:0/4003648979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab81c8810 con 0x7feab80b00c0 2023-10-06T13:53:23.352 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.345+0000 7feabef0a640 1 librados: init done 2023-10-06T13:53:23.353 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.346+0000 7feabef0a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:23.353 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.346+0000 7feabef0a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:23.353 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.346+0000 7feabef0a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:23.353 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.346+0000 7feabef0a640 1 -- 172.21.15.19:0/4003648979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab80af3d0 con 0x7feab80b00c0 2023-10-06T13:53:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.380+0000 7f4e05d9f640 10 monclient: get_monmap_and_config 2023-10-06T13:53:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.380+0000 7f4e05d9f640 10 monclient: build_initial_monmap 2023-10-06T13:53:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.380+0000 7f4e05d9f640 10 monclient: monmap: 2023-10-06T13:53:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:23.381102+0000 2023-10-06T13:53:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:23.381102+0000 2023-10-06T13:53:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:23.383 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.380+0000 7f4e05d9f640 1 Processor -- start 2023-10-06T13:53:23.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.380+0000 7f4e05d9f640 1 -- start start 2023-10-06T13:53:23.384 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.380+0000 7f4e05d9f640 10 monclient: init 2023-10-06T13:53:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.381+0000 7f4e05d9f640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.381+0000 7f4e05d9f640 10 monclient: _add_conns ranks=[3,0,1,4,2] 2023-10-06T13:53:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.381+0000 7f4e05d9f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4e000b3990 0x7f4e000af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.381+0000 7f4e05d9f640 10 monclient(hunting): picked mon.noname-a con 0x7f4e000b3990 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.381+0000 7f4e05d9f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4e000b3d60 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.381+0000 7f4e05d9f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4e000b00c0 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.381+0000 7f4e05d9f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e000aebe0 con 0x7f4e000b3d60 2023-10-06T13:53:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.381+0000 7f4e05d9f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e00141070 con 0x7f4e000b00c0 2023-10-06T13:53:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.381+0000 7f4e05d9f640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.381+0000 7f4e05d9f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e00140dd0 con 0x7f4e000b3990 2023-10-06T13:53:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.381+0000 7f4e05d9f640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.381+0000 7f4e05d9f640 10 monclient(hunting): authenticate will time out at 1331.567851s 2023-10-06T13:53:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.382+0000 7f4dff7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4e000b3990 0x7f4e000af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.382+0000 7f4dfeffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4e000b3d60 0x7f4e00141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58034/0 (socket says 172.21.15.19:58034) 2023-10-06T13:53:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.382+0000 7f4dfeffd640 1 -- 172.21.15.19:0/849436001 learned_addr learned my addr 172.21.15.19:0/849436001 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.382+0000 7f4dff7fe640 10 monclient(hunting): get_auth_request con 0x7f4e000b3990 auth_method 0 2023-10-06T13:53:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.382+0000 7f4dff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.382+0000 7f4dff7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.382+0000 7f4dff7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.383+0000 7f4dff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.383+0000 7f4dff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.383+0000 7f4dff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.385+0000 7f4dff7fe640 10 monclient(hunting): handle_auth_done global_id 4365 payload 274 2023-10-06T13:53:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.385+0000 7f4dff7fe640 1 -- 172.21.15.19:0/849436001 >> v1:172.21.15.83:6789/0 conn(0x7f4e000b00c0 legacy=0x7f4e0014b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.385+0000 7f4dff7fe640 1 -- 172.21.15.19:0/849436001 >> v1:172.21.15.19:6789/0 conn(0x7f4e000b3d60 legacy=0x7f4e00141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.385+0000 7f4dff7fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.385+0000 7f4dff7fe640 1 monclient: found mon.noname-a 2023-10-06T13:53:23.388 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.385+0000 7f4dff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:23.388 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.385+0000 7f4dff7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:23.388 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.385+0000 7f4dff7fe640 1 -- 172.21.15.19:0/849436001 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e00141c10 con 0x7f4e000b3990 2023-10-06T13:53:23.389 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.387+0000 7f4dff7fe640 1 --2- 172.21.15.19:0/849436001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e000b3990 0x7f4e000af810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4df00097a0 tx=0x7f4df002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73aedb48eac37d02 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:23.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4dfe7fc640 1 -- 172.21.15.19:0/849436001 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df002e070 con 0x7f4e000b3990 2023-10-06T13:53:23.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4dfe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4dfe7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:23.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4dfe7fc640 10 monclient: dump: 2023-10-06T13:53:23.396 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.396 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.396 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.396 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.396 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.396 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.396 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.396 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.396 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.396 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4dfe7fc640 10 monclient: _finish_auth 0 2023-10-06T13:53:23.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4dfe7fc640 10 monclient: _check_auth_tickets 2023-10-06T13:53:23.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4dfe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:23.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4dfe7fc640 1 -- 172.21.15.19:0/849436001 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df00315a0 con 0x7f4e000b3990 2023-10-06T13:53:23.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4dfe7fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:23.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4dfe7fc640 1 -- 172.21.15.19:0/849436001 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0031930 con 0x7f4e000b3990 2023-10-06T13:53:23.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4e05d9f640 5 monclient: authenticate success, global_id 4365 2023-10-06T13:53:23.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4e05d9f640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:23.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4e05d9f640 4 set_mon_vals no callback set 2023-10-06T13:53:23.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4dfe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4dfe7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:23.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.393+0000 7f4dfe7fc640 10 monclient: dump: 2023-10-06T13:53:23.397 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.398 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.398 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.398 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.398 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.398 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.398 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.398 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.398 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.398 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.394+0000 7f4e05d9f640 10 monclient: shutdown 2023-10-06T13:53:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.394+0000 7f4e05d9f640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.394+0000 7f4e05d9f640 1 -- 172.21.15.19:0/849436001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e000b3990 msgr2=0x7f4e000af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.394+0000 7f4e05d9f640 1 --2- 172.21.15.19:0/849436001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e000b3990 0x7f4e000af810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4df00097a0 tx=0x7f4df002b840 comp rx=0 tx=0).stop 2023-10-06T13:53:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.394+0000 7f4e05d9f640 1 -- 172.21.15.19:0/849436001 shutdown_connections 2023-10-06T13:53:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.394+0000 7f4e05d9f640 1 --2- 172.21.15.19:0/849436001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e000b3990 0x7f4e000af810 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.394+0000 7f4e05d9f640 1 -- 172.21.15.19:0/849436001 >> 172.21.15.19:0/849436001 conn(0x7f4e000b6e00 msgr2=0x7f4e000b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.394+0000 7f4e05d9f640 1 -- 172.21.15.19:0/849436001 shutdown_connections 2023-10-06T13:53:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.394+0000 7f4e05d9f640 1 -- 172.21.15.19:0/849436001 wait complete. 2023-10-06T13:53:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.395+0000 7f4e05d9f640 10 monclient: build_initial_monmap 2023-10-06T13:53:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.395+0000 7f4e05d9f640 10 monclient: monmap: 2023-10-06T13:53:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:23.403 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.403 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:23.403 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:23.403 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:23.403 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.395+0000 7f4e05d9f640 1 librados: starting msgr at 2023-10-06T13:53:23.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.395+0000 7f4e05d9f640 1 librados: starting objecter 2023-10-06T13:53:23.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.395+0000 7f4e05d9f640 1 Processor -- start 2023-10-06T13:53:23.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.395+0000 7f4e05d9f640 1 -- start start 2023-10-06T13:53:23.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.395+0000 7f4e05d9f640 1 librados: setting wanted keys 2023-10-06T13:53:23.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.395+0000 7f4e05d9f640 1 librados: calling monclient init 2023-10-06T13:53:23.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.395+0000 7f4e05d9f640 10 monclient: init 2023-10-06T13:53:23.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:23.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:53:23.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e000b00c0 0x7f4e00143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 10 monclient(hunting): picked mon.noname-a con 0x7f4e000b00c0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:23.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e000b3990 0x7f4e001cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 10 monclient(hunting): picked mon.noname-c con 0x7f4e000b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:23.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e000b3d60 0x7f4e00144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 10 monclient(hunting): picked mon.noname-b con 0x7f4e000b3d60 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:23.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e001446b0 con 0x7f4e000b00c0 2023-10-06T13:53:23.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e001cad80 con 0x7f4e000b3d60 2023-10-06T13:53:23.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e001cb060 con 0x7f4e000b3990 2023-10-06T13:53:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4e05d9f640 10 monclient(hunting): authenticate will time out at 1331.582786s 2023-10-06T13:53:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.396+0000 7f4dff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e000b00c0 0x7f4e00143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.397+0000 7f4dfeffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e000b3990 0x7f4e001cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.397+0000 7f4dff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e000b00c0 0x7f4e00143ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39138/0 (socket says 172.21.15.19:39138) 2023-10-06T13:53:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.397+0000 7f4dff7fe640 1 -- 172.21.15.19:0/1473246023 learned_addr learned my addr 172.21.15.19:0/1473246023 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.397+0000 7f4dff7fe640 10 monclient(hunting): get_auth_request con 0x7f4e000b00c0 auth_method 0 2023-10-06T13:53:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.397+0000 7f4dff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.397+0000 7f4dff7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.397+0000 7f4dff7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.397+0000 7f4dfeffd640 10 monclient(hunting): get_auth_request con 0x7f4e000b3990 auth_method 0 2023-10-06T13:53:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.397+0000 7f4dfeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.397+0000 7f4dfeffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.397+0000 7f4dfeffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.397+0000 7f4dff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.397+0000 7f4dff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.397+0000 7f4dff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.398+0000 7f4dfeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.398+0000 7f4dfeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.398+0000 7f4dfeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:23.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.402+0000 7f4dff7fe640 10 monclient(hunting): handle_auth_done global_id 4371 payload 824 2023-10-06T13:53:23.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.402+0000 7f4dff7fe640 1 -- 172.21.15.19:0/1473246023 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e000b3990 msgr2=0x7f4e001cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.402+0000 7f4dff7fe640 1 --2- 172.21.15.19:0/1473246023 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e000b3990 0x7f4e001cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.402+0000 7f4dff7fe640 1 -- 172.21.15.19:0/1473246023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e000b3d60 msgr2=0x7f4e00144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.402+0000 7f4dff7fe640 1 --2- 172.21.15.19:0/1473246023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e000b3d60 0x7f4e00144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.402+0000 7f4dff7fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:23.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.402+0000 7f4dff7fe640 1 monclient: found mon.noname-a 2023-10-06T13:53:23.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.402+0000 7f4dff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:23.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.402+0000 7f4dff7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:23.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.402+0000 7f4dff7fe640 1 -- 172.21.15.19:0/1473246023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df0009400 con 0x7f4e000b00c0 2023-10-06T13:53:23.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.405+0000 7f4dff7fe640 1 --2- 172.21.15.19:0/1473246023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e000b00c0 0x7f4e00143ec0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4df00097a0 tx=0x7f4df00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:23.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.405+0000 7f4de7fff640 10 client.?.objecter ms_handle_connect 0x7f4e000b00c0 2023-10-06T13:53:23.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.405+0000 7f4de7fff640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:23.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4de7fff640 1 -- 172.21.15.19:0/1473246023 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0032980 con 0x7f4e000b00c0 2023-10-06T13:53:23.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4de7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4de7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:23.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4de7fff640 10 monclient: dump: 2023-10-06T13:53:23.411 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.411 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.411 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.412 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.412 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.412 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.413 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.413 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.413 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.413 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4de7fff640 10 monclient: _finish_auth 0 2023-10-06T13:53:23.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4de7fff640 10 monclient: _check_auth_tickets 2023-10-06T13:53:23.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4de7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:23.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4de7fff640 1 -- 172.21.15.19:0/1473246023 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df0032ae0 con 0x7f4e000b00c0 2023-10-06T13:53:23.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4e05d9f640 5 monclient: authenticate success, global_id 4371 2023-10-06T13:53:23.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4e05d9f640 10 monclient: _renew_subs 2023-10-06T13:53:23.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4e05d9f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:23.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4e05d9f640 1 -- 172.21.15.19:0/1473246023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e001cb9c0 con 0x7f4e000b00c0 2023-10-06T13:53:23.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4de7fff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:23.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4de7fff640 1 -- 172.21.15.19:0/1473246023 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0032e90 con 0x7f4e000b00c0 2023-10-06T13:53:23.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4de7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4de7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:23.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4de7fff640 10 monclient: dump: 2023-10-06T13:53:23.414 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.414 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.414 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.414 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.414 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.415 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.415 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.415 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.415 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.415 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4dfdffb640 4 set_mon_vals no callback set 2023-10-06T13:53:23.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4e05d9f640 10 client.4371.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:23.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4e05d9f640 10 monclient: _renew_subs 2023-10-06T13:53:23.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4e05d9f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:23.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4e05d9f640 1 -- 172.21.15.19:0/1473246023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e00145080 con 0x7f4e000b00c0 2023-10-06T13:53:23.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.409+0000 7f4e05d9f640 1 librados: init done 2023-10-06T13:53:23.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.410+0000 7f4e05d9f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:23.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.410+0000 7f4e05d9f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:23.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.410+0000 7f4e05d9f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:23.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.410+0000 7f4e05d9f640 1 -- 172.21.15.19:0/1473246023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e000678e0 con 0x7f4e000b00c0 2023-10-06T13:53:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.429+0000 7f4de7fff640 1 -- 172.21.15.19:0/1473246023 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f4df004e470 con 0x7f4e000b00c0 2023-10-06T13:53:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.429+0000 7f4de7fff640 1 --2- 172.21.15.19:0/1473246023 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f4dec06c240 0x7f4dec06e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.430+0000 7f4dfeffd640 1 --2- 172.21.15.19:0/1473246023 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f4dec06c240 0x7f4dec06e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.430+0000 7f4dfeffd640 10 monclient: get_auth_request con 0x7f4dec06c240 auth_method 0 2023-10-06T13:53:23.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.430+0000 7f4de7fff640 10 client.4371.objecter ms_handle_connect 0x7f4dec06c240 2023-10-06T13:53:23.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.430+0000 7f4dfeffd640 1 --2- 172.21.15.19:0/1473246023 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f4dec06c240 0x7f4dec06e6f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4e001cd960 tx=0x7f4df4007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:23.483 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T13:53:23.588 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.586+0000 7feab57fa640 1 -- 172.21.15.19:0/4003648979 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7feaac003990 con 0x7feab80b00c0 2023-10-06T13:53:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.587+0000 7feab57fa640 1 --2- 172.21.15.19:0/4003648979 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fea9c06c210 0x7fea9c06e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.587+0000 7feab7fff640 1 --2- 172.21.15.19:0/4003648979 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fea9c06c210 0x7fea9c06e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.587+0000 7feab7fff640 10 monclient: get_auth_request con 0x7fea9c06c210 auth_method 0 2023-10-06T13:53:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.587+0000 7feab7fff640 1 --2- 172.21.15.19:0/4003648979 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fea9c06c210 0x7fea9c06e6c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feaa4036000 tx=0x7feaa403c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.587+0000 7feab57fa640 10 client.4384.objecter ms_handle_connect 0x7fea9c06c210 2023-10-06T13:53:23.590 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.588+0000 7feab57fa640 1 -- 172.21.15.19:0/4003648979 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7feaac057790 con 0x7feab80b00c0 2023-10-06T13:53:23.590 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.588+0000 7feab57fa640 10 client.4384.objecter ms_dispatch 0x7feab8000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:23.590 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.588+0000 7feab57fa640 3 client.4384.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:23.590 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.588+0000 7feab57fa640 3 client.4384.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:23.590 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.588+0000 7feab57fa640 20 client.4384.objecter dump_active .. 0 homeless 2023-10-06T13:53:23.591 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.589+0000 7fb01dd49640 10 monclient: get_monmap_and_config 2023-10-06T13:53:23.591 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.589+0000 7fb01dd49640 10 monclient: build_initial_monmap 2023-10-06T13:53:23.591 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.589+0000 7fb01dd49640 10 monclient: monmap: 2023-10-06T13:53:23.591 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:23.591 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.591 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:23.590614+0000 2023-10-06T13:53:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:23.590614+0000 2023-10-06T13:53:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:23.592 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 1 Processor -- start 2023-10-06T13:53:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 1 -- start start 2023-10-06T13:53:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 10 monclient: init 2023-10-06T13:53:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 10 monclient: _add_conns ranks=[1,4,2,0,3] 2023-10-06T13:53:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb0180b3990 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb0180b3d60 0x7fb0180af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 10 monclient(hunting): picked mon.noname-c con 0x7fb0180b3d60 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb0180b00c0 0x7fb0180b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 10 monclient(hunting): picked mon.noname-b con 0x7fb0180b00c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0180aebe0 con 0x7fb0180b3990 2023-10-06T13:53:23.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0180afd50 con 0x7fb0180b3d60 2023-10-06T13:53:23.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb018140dd0 con 0x7fb0180b00c0 2023-10-06T13:53:23.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:23.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.590+0000 7fb01dd49640 10 monclient(hunting): authenticate will time out at 1331.777323s 2023-10-06T13:53:23.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.591+0000 7fb017fff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb0180b00c0 0x7fb0180b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.591+0000 7fb017fff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb0180b00c0 0x7fb0180b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:41590/0 (socket says 172.21.15.19:41590) 2023-10-06T13:53:23.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.591+0000 7fb017fff640 1 -- 172.21.15.19:0/3945716589 learned_addr learned my addr 172.21.15.19:0/3945716589 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:23.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.591+0000 7fb017fff640 10 monclient(hunting): get_auth_request con 0x7fb0180b00c0 auth_method 0 2023-10-06T13:53:23.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.591+0000 7fb017fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.591+0000 7fb017fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.591+0000 7fb017fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.592+0000 7fb017fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.592+0000 7fb017fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.592+0000 7fb017fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.594+0000 7fb017fff640 10 monclient(hunting): handle_auth_done global_id 4304 payload 274 2023-10-06T13:53:23.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.594+0000 7fb017fff640 1 -- 172.21.15.19:0/3945716589 >> v2:172.21.15.83:3300/0 conn(0x7fb0180b3d60 msgr2=0x7fb0180af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:23.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.594+0000 7fb017fff640 1 --2- 172.21.15.19:0/3945716589 >> v2:172.21.15.83:3300/0 conn(0x7fb0180b3d60 0x7fb0180af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.594+0000 7fb017fff640 1 -- 172.21.15.19:0/3945716589 >> v1:172.21.15.83:6789/0 conn(0x7fb0180b3990 legacy=0x7fb018141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:23.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.594+0000 7fb017fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:23.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.594+0000 7fb017fff640 1 monclient: found mon.noname-b 2023-10-06T13:53:23.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.594+0000 7fb017fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:23.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.594+0000 7fb017fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T13:53:23.597 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.594+0000 7fb017fff640 1 -- 172.21.15.19:0/3945716589 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb018141070 con 0x7fb0180b00c0 2023-10-06T13:53:23.598 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.596+0000 7fb017fff640 1 --2- 172.21.15.19:0/3945716589 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb0180b00c0 0x7fb0180b0540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb010009e50 tx=0x7fb01002be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b269f793cc6ba21 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:23.600 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.599+0000 7feab57fa640 1 -- 172.21.15.19:0/4003648979 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7feaac053ad0 con 0x7feab80b00c0 2023-10-06T13:53:23.600 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.599+0000 7feab57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:23.600 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.599+0000 7feab57fa640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:23.603 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb0167fc640 1 -- 172.21.15.19:0/3945716589 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01002c9e0 con 0x7fb0180b00c0 2023-10-06T13:53:23.603 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb0167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.603 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb0167fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:23.603 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb0167fc640 10 monclient: dump: 2023-10-06T13:53:23.603 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.603 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.603 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.603 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.603 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.603 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.603 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.604 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.604 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.604 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.604 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb0167fc640 10 monclient: _finish_auth 0 2023-10-06T13:53:23.604 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb0167fc640 10 monclient: _check_auth_tickets 2023-10-06T13:53:23.604 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb0167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:23.604 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb0167fc640 1 -- 172.21.15.19:0/3945716589 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01002cb40 con 0x7fb0180b00c0 2023-10-06T13:53:23.604 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb01dd49640 5 monclient: authenticate success, global_id 4304 2023-10-06T13:53:23.604 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb01dd49640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:23.604 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb0167fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:23.604 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb01dd49640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:23.604 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb01dd49640 4 set_mon_vals no callback set 2023-10-06T13:53:23.605 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb0167fc640 1 -- 172.21.15.19:0/3945716589 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0100343f0 con 0x7fb0180b00c0 2023-10-06T13:53:23.605 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb0167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.605 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb0167fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:23.605 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.601+0000 7fb0167fc640 10 monclient: dump: 2023-10-06T13:53:23.605 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.605 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.605 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.605 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.605 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.605 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.605 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.605 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.605 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.606 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.602+0000 7fb01dd49640 10 monclient: shutdown 2023-10-06T13:53:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.602+0000 7fb01dd49640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.602+0000 7fb01dd49640 1 -- 172.21.15.19:0/3945716589 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb0180b00c0 msgr2=0x7fb0180b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.602+0000 7fb01dd49640 1 --2- 172.21.15.19:0/3945716589 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb0180b00c0 0x7fb0180b0540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb010009e50 tx=0x7fb01002be40 comp rx=0 tx=0).stop 2023-10-06T13:53:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.602+0000 7fb01dd49640 1 -- 172.21.15.19:0/3945716589 shutdown_connections 2023-10-06T13:53:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.602+0000 7fb01dd49640 1 --2- 172.21.15.19:0/3945716589 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb0180b00c0 0x7fb0180b0540 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.602+0000 7fb01dd49640 1 --2- 172.21.15.19:0/3945716589 >> v2:172.21.15.83:3300/0 conn(0x7fb0180b3d60 0x7fb0180af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.602+0000 7fb01dd49640 1 -- 172.21.15.19:0/3945716589 >> 172.21.15.19:0/3945716589 conn(0x7fb0180b6e00 msgr2=0x7fb0180b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.602+0000 7fb01dd49640 1 -- 172.21.15.19:0/3945716589 shutdown_connections 2023-10-06T13:53:23.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.602+0000 7fb01dd49640 1 -- 172.21.15.19:0/3945716589 wait complete. 2023-10-06T13:53:23.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.603+0000 7fb01dd49640 10 monclient: build_initial_monmap 2023-10-06T13:53:23.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.603+0000 7fb01dd49640 10 monclient: monmap: 2023-10-06T13:53:23.609 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:23.609 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:23.609 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:23.609 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:23.609 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:23.609 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.609 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:23.609 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:23.610 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:23.610 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.603+0000 7fb01dd49640 1 librados: starting msgr at 2023-10-06T13:53:23.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.603+0000 7fb01dd49640 1 librados: starting objecter 2023-10-06T13:53:23.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.603+0000 7fb01dd49640 1 Processor -- start 2023-10-06T13:53:23.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.603+0000 7fb01dd49640 1 -- start start 2023-10-06T13:53:23.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.603+0000 7fb01dd49640 1 librados: setting wanted keys 2023-10-06T13:53:23.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.603+0000 7fb01dd49640 1 librados: calling monclient init 2023-10-06T13:53:23.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.603+0000 7fb01dd49640 10 monclient: init 2023-10-06T13:53:23.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:23.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 10 monclient: _add_conns ranks=[2,0,1] 2023-10-06T13:53:23.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb0180b00c0 0x7fb018138a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 10 monclient(hunting): picked mon.noname-c con 0x7fb0180b00c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:23.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0180b3990 0x7fb018137de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 10 monclient(hunting): picked mon.noname-a con 0x7fb0180b3990 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:23.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0180b3d60 0x7fb018133760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 10 monclient(hunting): picked mon.noname-b con 0x7fb0180b3d60 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:23.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb018138470 con 0x7fb0180b3990 2023-10-06T13:53:23.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb018136e70 con 0x7fb0180b3d60 2023-10-06T13:53:23.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb018137030 con 0x7fb0180b00c0 2023-10-06T13:53:23.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:23.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.604+0000 7fb01dd49640 10 monclient(hunting): authenticate will time out at 1331.790578s 2023-10-06T13:53:23.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.605+0000 7fb0177fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb0180b00c0 0x7fb018138a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.606+0000 7fb0177fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb0180b00c0 0x7fb018138a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:41598/0 (socket says 172.21.15.19:41598) 2023-10-06T13:53:23.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.606+0000 7fb0177fe640 1 -- 172.21.15.19:0/927599858 learned_addr learned my addr 172.21.15.19:0/927599858 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:23.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.606+0000 7fb0177fe640 10 monclient(hunting): get_auth_request con 0x7fb0180b00c0 auth_method 0 2023-10-06T13:53:23.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.606+0000 7fb0177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:23.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.606+0000 7fb0177fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:23.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.606+0000 7fb0177fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:23.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.606+0000 7fb0177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:23.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.606+0000 7fb0177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:23.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.606+0000 7fb0177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:23.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.607+0000 7fb017fff640 1 --2- 172.21.15.19:0/927599858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0180b3d60 0x7fb018133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.608+0000 7fb017fff640 10 monclient(hunting): get_auth_request con 0x7fb0180b3d60 auth_method 0 2023-10-06T13:53:23.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.608+0000 7fb017fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:23.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.608+0000 7fb017fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:23.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.608+0000 7fb017fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:23.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.609+0000 7fb017fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:23.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.609+0000 7fb017fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:23.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.609+0000 7fb017fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:23.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.609+0000 7fb0177fe640 10 monclient(hunting): handle_auth_done global_id 4307 payload 824 2023-10-06T13:53:23.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.609+0000 7fb0177fe640 1 -- 172.21.15.19:0/927599858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0180b3d60 msgr2=0x7fb018133760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.609+0000 7fb0177fe640 1 --2- 172.21.15.19:0/927599858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0180b3d60 0x7fb018133760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.609+0000 7fb0177fe640 1 -- 172.21.15.19:0/927599858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0180b3990 msgr2=0x7fb018137de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.609+0000 7fb0177fe640 1 --2- 172.21.15.19:0/927599858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0180b3990 0x7fb018137de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.609+0000 7fb0177fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:23.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.609+0000 7fb0177fe640 1 monclient: found mon.noname-c 2023-10-06T13:53:23.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.609+0000 7fb0177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:23.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.609+0000 7fb0177fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T13:53:23.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.609+0000 7fb0177fe640 1 -- 172.21.15.19:0/927599858 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb010009b30 con 0x7fb0180b00c0 2023-10-06T13:53:23.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.611+0000 7fb0177fe640 1 --2- 172.21.15.19:0/927599858 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb0180b00c0 0x7fb018138a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c410 tx=0x7fb00800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:23.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.611+0000 7faffffff640 10 client.?.objecter ms_handle_connect 0x7fb0180b00c0 2023-10-06T13:53:23.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.611+0000 7faffffff640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7faffffff640 1 -- 172.21.15.19:0/927599858 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0080035f0 con 0x7fb0180b00c0 2023-10-06T13:53:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7faffffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7faffffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7faffffff640 10 monclient: dump: 2023-10-06T13:53:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.621 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.621 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.621 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.621 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.621 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.621 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7faffffff640 10 monclient: _finish_auth 0 2023-10-06T13:53:23.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7faffffff640 10 monclient: _check_auth_tickets 2023-10-06T13:53:23.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7faffffff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:23.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7faffffff640 1 -- 172.21.15.19:0/927599858 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb008003750 con 0x7fb0180b00c0 2023-10-06T13:53:23.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7faffffff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:23.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7faffffff640 1 -- 172.21.15.19:0/927599858 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008003b40 con 0x7fb0180b00c0 2023-10-06T13:53:23.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7fb01dd49640 5 monclient: authenticate success, global_id 4307 2023-10-06T13:53:23.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7fb01dd49640 10 monclient: _renew_subs 2023-10-06T13:53:23.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7fb01dd49640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:23.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7fb01dd49640 1 -- 172.21.15.19:0/927599858 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb018137310 con 0x7fb0180b00c0 2023-10-06T13:53:23.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7fb015ffb640 4 set_mon_vals no callback set 2023-10-06T13:53:23.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7faffffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7faffffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:23.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7faffffff640 10 monclient: dump: 2023-10-06T13:53:23.622 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.622 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.622 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.623 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.623 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.623 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.623 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.623 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.623 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.623 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7fb01dd49640 10 client.4307.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:23.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7fb01dd49640 10 monclient: _renew_subs 2023-10-06T13:53:23.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7fb01dd49640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:23.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7fb01dd49640 1 -- 172.21.15.19:0/927599858 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0181377d0 con 0x7fb0180b00c0 2023-10-06T13:53:23.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.618+0000 7fb01dd49640 1 librados: init done 2023-10-06T13:53:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.619+0000 7fb01dd49640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.619+0000 7fb01dd49640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.619+0000 7fb01dd49640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.619+0000 7fb01dd49640 1 -- 172.21.15.19:0/927599858 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0181379f0 con 0x7fb0180b00c0 2023-10-06T13:53:23.695 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T13:53:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.735+0000 7f4de7fff640 1 -- 172.21.15.19:0/1473246023 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4df007a6e0 con 0x7f4e000b00c0 2023-10-06T13:53:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.735+0000 7f4de7fff640 10 client.4371.objecter ms_dispatch 0x7f4e00000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.735+0000 7f4de7fff640 3 client.4371.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.735+0000 7f4de7fff640 3 client.4371.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.736+0000 7f4de7fff640 20 client.4371.objecter dump_active .. 0 homeless 2023-10-06T13:53:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.736+0000 7f4de7fff640 1 -- 172.21.15.19:0/1473246023 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f4df0076320 con 0x7f4e000b00c0 2023-10-06T13:53:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.736+0000 7f4de7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.736+0000 7f4de7fff640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:23.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.800+0000 7feabef0a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2023-10-06T13:53:23.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.800+0000 7feabef0a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2023-10-06T13:53:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.800+0000 7feabef0a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.800+0000 7feabef0a640 1 -- 172.21.15.19:0/4003648979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7feab80af3d0 con 0x7feab80b00c0 2023-10-06T13:53:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.804+0000 7feab57fa640 1 -- 172.21.15.19:0/4003648979 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feab80af3d0 con 0x7feab80b00c0 2023-10-06T13:53:23.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.804+0000 7feab57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2023-10-06T13:53:23.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.804+0000 7feab57fa640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:23.806 INFO:teuthology.orchestra.run.smithi019.stdout:64424509444 2023-10-06T13:53:23.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.805+0000 7feabef0a640 10 librados: watch_flush enter 2023-10-06T13:53:23.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.805+0000 7feabef0a640 10 librados: watch_flush exit 2023-10-06T13:53:23.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.805+0000 7feabef0a640 20 client.4384.objecter shutdown clearing up homeless session... 2023-10-06T13:53:23.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.805+0000 7feabef0a640 10 client.4384.objecter successfully canceled tick 2023-10-06T13:53:23.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 1 -- 172.21.15.19:0/4003648979 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fea9c06c210 msgr2=0x7fea9c06e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 1 --2- 172.21.15.19:0/4003648979 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fea9c06c210 0x7fea9c06e6c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feaa4036000 tx=0x7feaa403c040 comp rx=0 tx=0).stop 2023-10-06T13:53:23.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 10 monclient: shutdown 2023-10-06T13:53:23.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:23.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 1 -- 172.21.15.19:0/4003648979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab80b00c0 msgr2=0x7feab80ac300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 1 --2- 172.21.15.19:0/4003648979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab80b00c0 0x7feab80ac300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feaac00c390 tx=0x7feaac00c850 comp rx=0 tx=0).stop 2023-10-06T13:53:23.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 1 -- 172.21.15.19:0/4003648979 shutdown_connections 2023-10-06T13:53:23.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 1 --2- 172.21.15.19:0/4003648979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feab80b3d60 0x7feab81d5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 1 --2- 172.21.15.19:0/4003648979 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fea9c06c210 0x7fea9c06e6c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 1 --2- 172.21.15.19:0/4003648979 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7feab80b3990 0x7feab81d5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 1 --2- 172.21.15.19:0/4003648979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab80b00c0 0x7feab80ac300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 1 -- 172.21.15.19:0/4003648979 >> 172.21.15.19:0/4003648979 conn(0x7feab80b6e00 msgr2=0x7feab806b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:23.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 1 -- 172.21.15.19:0/4003648979 shutdown_connections 2023-10-06T13:53:23.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 1 -- 172.21.15.19:0/4003648979 wait complete. 2023-10-06T13:53:23.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.806+0000 7feabef0a640 1 librados: shutdown 2023-10-06T13:53:23.819 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.2 2023-10-06T13:53:23.819 DEBUG:teuthology.parallel:result is None 2023-10-06T13:53:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.820+0000 7fe3d36a4640 10 monclient: get_monmap_and_config 2023-10-06T13:53:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.820+0000 7fe3d36a4640 10 monclient: build_initial_monmap 2023-10-06T13:53:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.820+0000 7fe3d36a4640 10 monclient: monmap: 2023-10-06T13:53:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:23.821524+0000 2023-10-06T13:53:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:23.821524+0000 2023-10-06T13:53:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:23.823 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 1 Processor -- start 2023-10-06T13:53:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 1 -- start start 2023-10-06T13:53:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 10 monclient: init 2023-10-06T13:53:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 10 monclient: _add_conns ranks=[3,1,0,2,4] 2023-10-06T13:53:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe3cc0b3990 0x7fe3cc0af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 10 monclient(hunting): picked mon.noname-a con 0x7fe3cc0b3990 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe3cc0b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe3cc0b00c0 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3cc0aebe0 con 0x7fe3cc0b00c0 2023-10-06T13:53:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3cc141070 con 0x7fe3cc0b3d60 2023-10-06T13:53:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3cc140dd0 con 0x7fe3cc0b3990 2023-10-06T13:53:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.821+0000 7fe3d36a4640 10 monclient(hunting): authenticate will time out at 1332.008284s 2023-10-06T13:53:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.822+0000 7fe3d0c3f640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe3cc0b3d60 0x7fe3cc141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:38470/0 (socket says 172.21.15.19:38470) 2023-10-06T13:53:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.822+0000 7fe3d0c3f640 1 -- 172.21.15.19:0/2346463970 learned_addr learned my addr 172.21.15.19:0/2346463970 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.822+0000 7fe3d1440640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe3cc0b3990 0x7fe3cc0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.823+0000 7fe3d1440640 10 monclient(hunting): get_auth_request con 0x7fe3cc0b3990 auth_method 0 2023-10-06T13:53:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.823+0000 7fe3d1440640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.823+0000 7fe3d1440640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.823+0000 7fe3d1440640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.823+0000 7fe3d1440640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.823+0000 7fe3d1440640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.823+0000 7fe3d1440640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.826+0000 7fe3d1440640 10 monclient(hunting): handle_auth_done global_id 4374 payload 274 2023-10-06T13:53:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.826+0000 7fe3d1440640 1 -- 172.21.15.19:0/2346463970 >> v1:172.21.15.83:6789/0 conn(0x7fe3cc0b3d60 legacy=0x7fe3cc141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.826+0000 7fe3d1440640 1 -- 172.21.15.19:0/2346463970 >> v1:172.21.15.19:6789/0 conn(0x7fe3cc0b00c0 legacy=0x7fe3cc14b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.826+0000 7fe3d1440640 10 monclient: _finish_hunting 0 2023-10-06T13:53:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.826+0000 7fe3d1440640 1 monclient: found mon.noname-a 2023-10-06T13:53:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.826+0000 7fe3d1440640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.826+0000 7fe3d1440640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.826+0000 7fe3d1440640 1 -- 172.21.15.19:0/2346463970 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc141c10 con 0x7fe3cc0b3990 2023-10-06T13:53:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.829+0000 7fe3d1440640 1 --2- 172.21.15.19:0/2346463970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc0b3990 0x7fe3cc0af810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80097a0 tx=0x7fe3c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc14ea1711d87f69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:23.837 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3c3fff640 1 -- 172.21.15.19:0/2346463970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8031830 con 0x7fe3cc0b3990 2023-10-06T13:53:23.837 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.837 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3c3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:23.837 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3c3fff640 10 monclient: dump: 2023-10-06T13:53:23.837 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.837 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.837 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.837 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.837 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.837 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.838 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3c3fff640 10 monclient: _finish_auth 0 2023-10-06T13:53:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3c3fff640 10 monclient: _check_auth_tickets 2023-10-06T13:53:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3c3fff640 1 -- 172.21.15.19:0/2346463970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c8031990 con 0x7fe3cc0b3990 2023-10-06T13:53:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3c3fff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3c3fff640 1 -- 172.21.15.19:0/2346463970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8031d20 con 0x7fe3cc0b3990 2023-10-06T13:53:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3c3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3c3fff640 10 monclient: dump: 2023-10-06T13:53:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.839 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3d36a4640 5 monclient: authenticate success, global_id 4374 2023-10-06T13:53:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3d36a4640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3d36a4640 4 set_mon_vals no callback set 2023-10-06T13:53:23.842 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3d36a4640 10 monclient: shutdown 2023-10-06T13:53:23.842 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3d36a4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:23.842 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3d36a4640 1 -- 172.21.15.19:0/2346463970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc0b3990 msgr2=0x7fe3cc0af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.842 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.835+0000 7fe3d36a4640 1 --2- 172.21.15.19:0/2346463970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc0b3990 0x7fe3cc0af810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80097a0 tx=0x7fe3c802b8d0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.836+0000 7fe3d36a4640 1 -- 172.21.15.19:0/2346463970 shutdown_connections 2023-10-06T13:53:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.836+0000 7fe3d36a4640 1 --2- 172.21.15.19:0/2346463970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc0b3990 0x7fe3cc0af810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.836+0000 7fe3d36a4640 1 -- 172.21.15.19:0/2346463970 >> 172.21.15.19:0/2346463970 conn(0x7fe3cc0b6e00 msgr2=0x7fe3cc0b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.836+0000 7fe3d36a4640 1 -- 172.21.15.19:0/2346463970 shutdown_connections 2023-10-06T13:53:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.836+0000 7fe3d36a4640 1 -- 172.21.15.19:0/2346463970 wait complete. 2023-10-06T13:53:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.836+0000 7fe3d36a4640 10 monclient: build_initial_monmap 2023-10-06T13:53:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.836+0000 7fe3d36a4640 10 monclient: monmap: 2023-10-06T13:53:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:23.844 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.844 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:23.844 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:23.844 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:23.844 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.844 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 1 librados: starting msgr at 2023-10-06T13:53:23.844 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 1 librados: starting objecter 2023-10-06T13:53:23.844 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 1 Processor -- start 2023-10-06T13:53:23.844 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 1 -- start start 2023-10-06T13:53:23.844 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 1 librados: setting wanted keys 2023-10-06T13:53:23.844 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 1 librados: calling monclient init 2023-10-06T13:53:23.844 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 10 monclient: init 2023-10-06T13:53:23.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:23.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T13:53:23.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3cc0b00c0 0x7fe3cc13c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 10 monclient(hunting): picked mon.noname-b con 0x7fe3cc0b00c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:23.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe3cc0b3990 0x7fe3cc143e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 10 monclient(hunting): picked mon.noname-c con 0x7fe3cc0b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:23.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc0b3d60 0x7fe3cc1420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 10 monclient(hunting): picked mon.noname-a con 0x7fe3cc0b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:23.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc144f10 con 0x7fe3cc0b3d60 2023-10-06T13:53:23.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc0a95e0 con 0x7fe3cc0b00c0 2023-10-06T13:53:23.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:23.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc0a8450 con 0x7fe3cc0b3990 2023-10-06T13:53:23.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:23.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.837+0000 7fe3d36a4640 10 monclient(hunting): authenticate will time out at 1332.024214s 2023-10-06T13:53:23.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.838+0000 7fe3d1440640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3cc0b00c0 0x7fe3cc13c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.838+0000 7fe3d1c41640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc0b3d60 0x7fe3cc1420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.838+0000 7fe3d1440640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3cc0b00c0 0x7fe3cc13c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39670/0 (socket says 172.21.15.19:39670) 2023-10-06T13:53:23.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.838+0000 7fe3d1440640 1 -- 172.21.15.19:0/1630430231 learned_addr learned my addr 172.21.15.19:0/1630430231 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:23.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.838+0000 7fe3d1440640 10 monclient(hunting): get_auth_request con 0x7fe3cc0b00c0 auth_method 0 2023-10-06T13:53:23.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.838+0000 7fe3d1440640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:23.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.838+0000 7fe3d1440640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:23.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.838+0000 7fe3d1440640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:23.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.839+0000 7fe3d1c41640 10 monclient(hunting): get_auth_request con 0x7fe3cc0b3d60 auth_method 0 2023-10-06T13:53:23.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.839+0000 7fe3d1c41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:23.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.839+0000 7fe3d1c41640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:23.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.839+0000 7fe3d1c41640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:23.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.839+0000 7fe3d1440640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:23.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.839+0000 7fe3d1440640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:23.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.839+0000 7fe3d1440640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:23.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.839+0000 7fe3d1c41640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.839+0000 7fe3d1c41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.839+0000 7fe3d1c41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.842+0000 7fe3d1440640 10 monclient(hunting): handle_auth_done global_id 4390 payload 824 2023-10-06T13:53:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.842+0000 7fe3d1440640 1 -- 172.21.15.19:0/1630430231 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe3cc0b3990 msgr2=0x7fe3cc143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.842+0000 7fe3d1440640 1 --2- 172.21.15.19:0/1630430231 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe3cc0b3990 0x7fe3cc143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.842+0000 7fe3d1440640 1 -- 172.21.15.19:0/1630430231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc0b3d60 msgr2=0x7fe3cc1420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.842+0000 7fe3d1440640 1 --2- 172.21.15.19:0/1630430231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc0b3d60 0x7fe3cc1420d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.842+0000 7fe3d1440640 10 monclient: _finish_hunting 0 2023-10-06T13:53:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.842+0000 7fe3d1440640 1 monclient: found mon.noname-b 2023-10-06T13:53:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.842+0000 7fe3d1440640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.842+0000 7fe3d1440640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:53:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.842+0000 7fe3d1440640 1 -- 172.21.15.19:0/1630430231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c8009400 con 0x7fe3cc0b00c0 2023-10-06T13:53:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.844+0000 7fe3d1440640 1 --2- 172.21.15.19:0/1630430231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3cc0b00c0 0x7fe3cc13c2a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3c802bee0 tx=0x7fe3c80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.844+0000 7fe3c1ffb640 10 client.?.objecter ms_handle_connect 0x7fe3cc0b00c0 2023-10-06T13:53:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.844+0000 7fe3c1ffb640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3c1ffb640 1 -- 172.21.15.19:0/1630430231 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c803ba70 con 0x7fe3cc0b00c0 2023-10-06T13:53:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3c1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3c1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3c1ffb640 10 monclient: dump: 2023-10-06T13:53:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.852 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.852 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.852 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.852 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.852 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.852 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3c1ffb640 10 monclient: _finish_auth 0 2023-10-06T13:53:23.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3c1ffb640 10 monclient: _check_auth_tickets 2023-10-06T13:53:23.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3c1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:23.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3c1ffb640 1 -- 172.21.15.19:0/1630430231 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c803bbf0 con 0x7fe3cc0b00c0 2023-10-06T13:53:23.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3d36a4640 5 monclient: authenticate success, global_id 4390 2023-10-06T13:53:23.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3d36a4640 10 monclient: _renew_subs 2023-10-06T13:53:23.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3d36a4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:23.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3d36a4640 1 -- 172.21.15.19:0/1630430231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3cc143210 con 0x7fe3cc0b00c0 2023-10-06T13:53:23.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3c1ffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:23.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3c1ffb640 1 -- 172.21.15.19:0/1630430231 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80453f0 con 0x7fe3cc0b00c0 2023-10-06T13:53:23.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3c1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:23.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3c1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:53:23.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3c1ffb640 10 monclient: dump: 2023-10-06T13:53:23.853 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:23.853 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:23.854 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.854 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:23.854 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:23.854 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:23.854 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:23.854 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:23.854 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:23.854 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:23.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.847+0000 7fe3c37fe640 4 set_mon_vals no callback set 2023-10-06T13:53:23.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.848+0000 7fe3d36a4640 10 client.4390.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:23.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.848+0000 7fe3d36a4640 10 monclient: _renew_subs 2023-10-06T13:53:23.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.848+0000 7fe3d36a4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:23.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.848+0000 7fe3d36a4640 1 -- 172.21.15.19:0/1630430231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3cc1d9790 con 0x7fe3cc0b00c0 2023-10-06T13:53:23.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.848+0000 7fe3d36a4640 1 librados: init done 2023-10-06T13:53:23.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.849+0000 7fe3d36a4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:23.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.849+0000 7fe3d36a4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:23.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.849+0000 7fe3d36a4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:23.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.849+0000 7fe3d36a4640 1 -- 172.21.15.19:0/1630430231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3cc0678e0 con 0x7fe3cc0b00c0 2023-10-06T13:53:23.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.915+0000 7faffffff640 1 -- 172.21.15.19:0/927599858 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb008003ca0 con 0x7fb0180b00c0 2023-10-06T13:53:23.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.915+0000 7faffffff640 1 --2- 172.21.15.19:0/927599858 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fb00406c1c0 0x7fb00406e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:23.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.916+0000 7fb016ffd640 1 --2- 172.21.15.19:0/927599858 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fb00406c1c0 0x7fb00406e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:23.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.916+0000 7fb016ffd640 10 monclient: get_auth_request con 0x7fb00406c1c0 auth_method 0 2023-10-06T13:53:23.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.916+0000 7fb016ffd640 1 --2- 172.21.15.19:0/927599858 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fb00406c1c0 0x7fb00406e670 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb018135920 tx=0x7fb00c006bf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:23.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.916+0000 7faffffff640 10 client.4307.objecter ms_handle_connect 0x7fb00406c1c0 2023-10-06T13:53:23.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.928+0000 7faffffff640 1 -- 172.21.15.19:0/927599858 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb0080010c0 con 0x7fb0180b00c0 2023-10-06T13:53:23.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.928+0000 7faffffff640 10 client.4307.objecter ms_dispatch 0x7fb018000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:23.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.928+0000 7faffffff640 3 client.4307.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:23.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.928+0000 7faffffff640 3 client.4307.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:23.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.928+0000 7faffffff640 20 client.4307.objecter dump_active .. 0 homeless 2023-10-06T13:53:23.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.928+0000 7faffffff640 1 -- 172.21.15.19:0/927599858 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7fb0080536b0 con 0x7fb0180b00c0 2023-10-06T13:53:23.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.928+0000 7faffffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:23.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.928+0000 7faffffff640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:23.941 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T13:53:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.950+0000 7f4e05d9f640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2023-10-06T13:53:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.950+0000 7f4e05d9f640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2023-10-06T13:53:23.952 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.950+0000 7f4e05d9f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:23.952 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:23.950+0000 7f4e05d9f640 1 -- 172.21.15.19:0/1473246023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4e000ad9f0 con 0x7f4e000b00c0 2023-10-06T13:53:24.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T13:53:24.034 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.032+0000 7f4de7fff640 1 -- 172.21.15.19:0/1473246023 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4df0079940 con 0x7f4e000b00c0 2023-10-06T13:53:24.034 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.032+0000 7f4de7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2023-10-06T13:53:24.034 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.032+0000 7f4de7fff640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:24.035 INFO:teuthology.orchestra.run.smithi019.stdout:68719476743 2023-10-06T13:53:24.037 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 10 librados: watch_flush enter 2023-10-06T13:53:24.037 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 10 librados: watch_flush exit 2023-10-06T13:53:24.037 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 20 client.4371.objecter shutdown clearing up homeless session... 2023-10-06T13:53:24.037 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 10 client.4371.objecter successfully canceled tick 2023-10-06T13:53:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 1 -- 172.21.15.19:0/1473246023 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f4dec06c240 msgr2=0x7f4dec06e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 1 --2- 172.21.15.19:0/1473246023 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f4dec06c240 0x7f4dec06e6f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4e001cd960 tx=0x7f4df4007e80 comp rx=0 tx=0).stop 2023-10-06T13:53:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 10 monclient: shutdown 2023-10-06T13:53:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 1 -- 172.21.15.19:0/1473246023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e000b00c0 msgr2=0x7f4e00143ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 1 --2- 172.21.15.19:0/1473246023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e000b00c0 0x7f4e00143ec0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4df00097a0 tx=0x7f4df00046c0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 1 -- 172.21.15.19:0/1473246023 shutdown_connections 2023-10-06T13:53:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 1 --2- 172.21.15.19:0/1473246023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e000b3d60 0x7f4e00144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 1 --2- 172.21.15.19:0/1473246023 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f4dec06c240 0x7f4dec06e6f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 1 --2- 172.21.15.19:0/1473246023 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e000b3990 0x7f4e001cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 1 --2- 172.21.15.19:0/1473246023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e000b00c0 0x7f4e00143ec0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 1 -- 172.21.15.19:0/1473246023 >> 172.21.15.19:0/1473246023 conn(0x7f4e000b6e00 msgr2=0x7f4e0006b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.035+0000 7f4e05d9f640 1 -- 172.21.15.19:0/1473246023 shutdown_connections 2023-10-06T13:53:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.036+0000 7f4e05d9f640 1 -- 172.21.15.19:0/1473246023 wait complete. 2023-10-06T13:53:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.036+0000 7f4e05d9f640 1 librados: shutdown 2023-10-06T13:53:24.057 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476743 for osd.1 2023-10-06T13:53:24.057 DEBUG:teuthology.parallel:result is None 2023-10-06T13:53:24.064 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.063+0000 7f4e8cfc4640 10 monclient: get_monmap_and_config 2023-10-06T13:53:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.063+0000 7f4e8cfc4640 10 monclient: build_initial_monmap 2023-10-06T13:53:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.063+0000 7f4e8cfc4640 10 monclient: monmap: 2023-10-06T13:53:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:24.064136+0000 2023-10-06T13:53:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:24.064136+0000 2023-10-06T13:53:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:24.066 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:24.066 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.063+0000 7f4e8cfc4640 1 Processor -- start 2023-10-06T13:53:24.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.063+0000 7f4e8cfc4640 1 -- start start 2023-10-06T13:53:24.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.063+0000 7f4e8cfc4640 10 monclient: init 2023-10-06T13:53:24.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.064+0000 7f4e8cfc4640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:24.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.064+0000 7f4e8cfc4640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2023-10-06T13:53:24.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.064+0000 7f4e8cfc4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4e880b3990 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:24.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.064+0000 7f4e8cfc4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e880b3d60 0x7f4e880af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.064+0000 7f4e8cfc4640 10 monclient(hunting): picked mon.noname-c con 0x7f4e880b3d60 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:24.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.064+0000 7f4e8cfc4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4e880b00c0 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:24.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.064+0000 7f4e8cfc4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e880aebe0 con 0x7f4e880b00c0 2023-10-06T13:53:24.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.064+0000 7f4e8cfc4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e88141070 con 0x7f4e880b3990 2023-10-06T13:53:24.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.064+0000 7f4e8cfc4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.064+0000 7f4e8cfc4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e88140dd0 con 0x7f4e880b3d60 2023-10-06T13:53:24.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.064+0000 7f4e8cfc4640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:24.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.064+0000 7f4e8cfc4640 10 monclient(hunting): authenticate will time out at 1332.251042s 2023-10-06T13:53:24.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.065+0000 7f4e86d9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4e880b00c0 0x7f4e8814b600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58058/0 (socket says 172.21.15.19:58058) 2023-10-06T13:53:24.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.065+0000 7f4e86d9d640 1 -- 172.21.15.19:0/1078059460 learned_addr learned my addr 172.21.15.19:0/1078059460 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.067+0000 7f4e8559a640 1 -- 172.21.15.19:0/1078059460 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697827072 0 0) 0x7f4e880aebe0 con 0x7f4e880b00c0 2023-10-06T13:53:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.067+0000 7f4e8559a640 10 monclient(hunting): _negotiate 2023-10-06T13:53:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.067+0000 7f4e8559a640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.067+0000 7f4e8559a640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.067+0000 7f4e8559a640 10 monclient(hunting): my global_id is 4380 2023-10-06T13:53:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.067+0000 7f4e8559a640 1 -- 172.21.15.19:0/1078059460 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e70003520 con 0x7f4e880b00c0 2023-10-06T13:53:24.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 -- 172.21.15.19:0/1078059460 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2514754728 0 0) 0x7f4e70003520 con 0x7f4e880b00c0 2023-10-06T13:53:24.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 -- 172.21.15.19:0/1078059460 >> v2:172.21.15.83:3300/0 conn(0x7f4e880b3d60 msgr2=0x7f4e880af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:24.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 --2- 172.21.15.19:0/1078059460 >> v2:172.21.15.83:3300/0 conn(0x7f4e880b3d60 0x7f4e880af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 -- 172.21.15.19:0/1078059460 >> v1:172.21.15.83:6789/0 conn(0x7f4e880b3990 legacy=0x7f4e88141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:24.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient: _finish_hunting 0 2023-10-06T13:53:24.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 monclient: found mon.noname-a-legacy 2023-10-06T13:53:24.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:24.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.19:6789/0 2023-10-06T13:53:24.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 -- 172.21.15.19:0/1078059460 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e88141c10 con 0x7f4e880b00c0 2023-10-06T13:53:24.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient: _finish_auth 0 2023-10-06T13:53:24.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient: _check_auth_tickets 2023-10-06T13:53:24.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:24.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 -- 172.21.15.19:0/1078059460 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251531364 0 0) 0x7f4e800019f0 con 0x7f4e880b00c0 2023-10-06T13:53:24.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:24.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2023-10-06T13:53:24.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient: dump: 2023-10-06T13:53:24.074 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:24.075 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.075 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.075 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.075 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:24.075 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.075 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:24.075 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:24.075 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:24.075 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.075 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] but i'm connected to v1:172.21.15.19:6789/0, reconnecting 2023-10-06T13:53:24.075 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:24.075 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 -- 172.21.15.19:0/1078059460 >> v1:172.21.15.19:6789/0 conn(0x7f4e880b00c0 legacy=0x7f4e8814b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient: _add_conns ranks=[2,0,1] 2023-10-06T13:53:24.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 --2- 172.21.15.19:0/1078059460 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e70003970 0x7f4e70003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient(hunting): picked mon.c con 0x7f4e70003970 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:24.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 --2- 172.21.15.19:0/1078059460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e700047c0 0x7f4e70004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient(hunting): picked mon.a con 0x7f4e700047c0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:24.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 --2- 172.21.15.19:0/1078059460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e700052c0 0x7f4e7000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient(hunting): picked mon.b con 0x7f4e700052c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:24.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 -- 172.21.15.19:0/1078059460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e88140dd0 con 0x7f4e700047c0 2023-10-06T13:53:24.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 -- 172.21.15.19:0/1078059460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7000c1f0 con 0x7f4e700052c0 2023-10-06T13:53:24.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 1 -- 172.21.15.19:0/1078059460 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7000c3b0 con 0x7f4e70003970 2023-10-06T13:53:24.077 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:24.077 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8559a640 10 monclient(hunting): _finish_auth 0 2023-10-06T13:53:24.077 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.071+0000 7f4e8659c640 1 --2- 172.21.15.19:0/1078059460 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e70003970 0x7f4e70003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:24.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.072+0000 7f4e8659c640 10 monclient(hunting): get_auth_request con 0x7f4e70003970 auth_method 0 2023-10-06T13:53:24.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.072+0000 7f4e8659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:24.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.072+0000 7f4e8659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:24.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.072+0000 7f4e8659c640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T13:53:24.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.072+0000 7f4e8659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:24.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.072+0000 7f4e8659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:24.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.072+0000 7f4e8659c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T13:53:24.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.074+0000 7f4e8659c640 10 monclient(hunting): handle_auth_done global_id 4380 payload 293 2023-10-06T13:53:24.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.074+0000 7f4e8659c640 1 -- 172.21.15.19:0/1078059460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e700052c0 msgr2=0x7f4e7000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:24.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.074+0000 7f4e8659c640 1 --2- 172.21.15.19:0/1078059460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e700052c0 0x7f4e7000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.074+0000 7f4e8659c640 1 -- 172.21.15.19:0/1078059460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e700047c0 msgr2=0x7f4e70004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:24.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.074+0000 7f4e8659c640 1 --2- 172.21.15.19:0/1078059460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e700047c0 0x7f4e70004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.075+0000 7f4e8659c640 10 monclient: _finish_hunting 0 2023-10-06T13:53:24.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.075+0000 7f4e8659c640 1 monclient: found mon.c 2023-10-06T13:53:24.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.075+0000 7f4e86d9d640 1 -- 172.21.15.19:0/1078059460 reap_dead start 2023-10-06T13:53:24.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.075+0000 7f4e8659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:24.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.075+0000 7f4e8659c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:24.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.075+0000 7f4e8659c640 1 -- 172.21.15.19:0/1078059460 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e70003520 con 0x7f4e70003970 2023-10-06T13:53:24.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.075+0000 7f4e8659c640 10 monclient: _finish_auth 0 2023-10-06T13:53:24.080 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.075+0000 7f4e8659c640 10 monclient: _check_auth_tickets 2023-10-06T13:53:24.080 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.075+0000 7f4e8659c640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:24.080 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.075+0000 7f4e8cfc4640 5 monclient: authenticate success, global_id 4380 2023-10-06T13:53:24.080 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.075+0000 7f4e8cfc4640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:24.080 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.077+0000 7f4e8659c640 1 --2- 172.21.15.19:0/1078059460 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e70003970 0x7f4e70003d60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e78001650 tx=0x7f4e700057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8721ee036cc98c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.081+0000 7f4e8559a640 1 -- 172.21.15.19:0/1078059460 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e78005270 con 0x7f4e70003970 2023-10-06T13:53:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.081+0000 7f4e8559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.081+0000 7f4e8559a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.081+0000 7f4e8559a640 10 monclient: dump: 2023-10-06T13:53:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:24.084 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.081+0000 7f4e8559a640 1 -- 172.21.15.19:0/1078059460 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7802e070 con 0x7f4e70003970 2023-10-06T13:53:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.081+0000 7f4e8cfc4640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.081+0000 7f4e8559a640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.081+0000 7f4e8cfc4640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.081+0000 7f4e8cfc4640 4 set_mon_vals no callback set 2023-10-06T13:53:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.081+0000 7f4e8559a640 1 -- 172.21.15.19:0/1078059460 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e78003610 con 0x7f4e70003970 2023-10-06T13:53:24.085 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.081+0000 7f4e8559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:24.085 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.081+0000 7f4e8559a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:24.085 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.081+0000 7f4e8559a640 10 monclient: dump: 2023-10-06T13:53:24.085 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:24.085 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.085 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.085 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.085 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:24.085 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.085 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:24.085 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:24.085 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:24.086 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.088 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.082+0000 7f4e8cfc4640 10 monclient: shutdown 2023-10-06T13:53:24.088 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.082+0000 7f4e8cfc4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:24.088 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.082+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/1078059460 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e70003970 msgr2=0x7f4e70003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.088 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.082+0000 7f4e8cfc4640 1 --2- 172.21.15.19:0/1078059460 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e70003970 0x7f4e70003d60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e78001650 tx=0x7f4e700057f0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.088 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.082+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/1078059460 shutdown_connections 2023-10-06T13:53:24.088 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.082+0000 7f4e8cfc4640 1 --2- 172.21.15.19:0/1078059460 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e70003970 0x7f4e70003d60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.088 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.082+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/1078059460 >> 172.21.15.19:0/1078059460 conn(0x7f4e880b6e00 msgr2=0x7f4e880b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:24.088 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.082+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/1078059460 shutdown_connections 2023-10-06T13:53:24.089 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.082+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/1078059460 wait complete. 2023-10-06T13:53:24.089 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.083+0000 7f4e8cfc4640 10 monclient: build_initial_monmap 2023-10-06T13:53:24.089 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.083+0000 7f4e8cfc4640 10 monclient: monmap: 2023-10-06T13:53:24.089 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:24.089 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:24.089 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:24.089 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:24.089 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:24.089 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.089 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:24.089 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:24.089 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:24.090 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.083+0000 7f4e8cfc4640 1 librados: starting msgr at 2023-10-06T13:53:24.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.083+0000 7f4e8cfc4640 1 librados: starting objecter 2023-10-06T13:53:24.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.083+0000 7f4e8cfc4640 1 Processor -- start 2023-10-06T13:53:24.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 1 -- start start 2023-10-06T13:53:24.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 1 librados: setting wanted keys 2023-10-06T13:53:24.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 1 librados: calling monclient init 2023-10-06T13:53:24.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 10 monclient: init 2023-10-06T13:53:24.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:24.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T13:53:24.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e70003970 0x7f4e880a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 10 monclient(hunting): picked mon.noname-a con 0x7f4e70003970 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e880aa530 0x7f4e88144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 10 monclient(hunting): picked mon.noname-b con 0x7f4e880aa530 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e88139f40 0x7f4e8813a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 10 monclient(hunting): picked mon.noname-c con 0x7f4e88139f40 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880aabe0 con 0x7f4e70003970 2023-10-06T13:53:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e88144ec0 con 0x7f4e880aa530 2023-10-06T13:53:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880ab000 con 0x7f4e88139f40 2023-10-06T13:53:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.084+0000 7f4e8cfc4640 10 monclient(hunting): authenticate will time out at 1332.270976s 2023-10-06T13:53:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.085+0000 7f4e86d9d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e88139f40 0x7f4e8813a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.085+0000 7f4e8659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e70003970 0x7f4e880a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.085+0000 7f4e86d9d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e88139f40 0x7f4e8813a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:41624/0 (socket says 172.21.15.19:41624) 2023-10-06T13:53:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.085+0000 7f4e86d9d640 1 -- 172.21.15.19:0/182689881 learned_addr learned my addr 172.21.15.19:0/182689881 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.085+0000 7f4e86d9d640 10 monclient(hunting): get_auth_request con 0x7f4e88139f40 auth_method 0 2023-10-06T13:53:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.085+0000 7f4e86d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.085+0000 7f4e86d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.085+0000 7f4e86d9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.085+0000 7f4e8659c640 10 monclient(hunting): get_auth_request con 0x7f4e70003970 auth_method 0 2023-10-06T13:53:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.085+0000 7f4e8659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.085+0000 7f4e8659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.085+0000 7f4e8659c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.086+0000 7f4e86d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.086+0000 7f4e86d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.086+0000 7f4e86d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.086+0000 7f4e8659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.086+0000 7f4e8659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.086+0000 7f4e8659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.089+0000 7f4e86d9d640 10 monclient(hunting): handle_auth_done global_id 4310 payload 824 2023-10-06T13:53:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.089+0000 7f4e86d9d640 1 -- 172.21.15.19:0/182689881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e880aa530 msgr2=0x7f4e88144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.089+0000 7f4e86d9d640 1 --2- 172.21.15.19:0/182689881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e880aa530 0x7f4e88144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.089+0000 7f4e86d9d640 1 -- 172.21.15.19:0/182689881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e70003970 msgr2=0x7f4e880a9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.089+0000 7f4e86d9d640 1 --2- 172.21.15.19:0/182689881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e70003970 0x7f4e880a9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.089+0000 7f4e86d9d640 10 monclient: _finish_hunting 0 2023-10-06T13:53:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.089+0000 7f4e86d9d640 1 monclient: found mon.noname-c 2023-10-06T13:53:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.089+0000 7f4e86d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.089+0000 7f4e86d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T13:53:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.089+0000 7f4e86d9d640 1 -- 172.21.15.19:0/182689881 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8813a8f0 con 0x7f4e88139f40 2023-10-06T13:53:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.091+0000 7f4e86d9d640 1 --2- 172.21.15.19:0/182689881 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e88139f40 0x7f4e8813a3b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4e80000f80 tx=0x7f4e8000ca70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.091+0000 7f4e76ffd640 10 client.?.objecter ms_handle_connect 0x7f4e88139f40 2023-10-06T13:53:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.091+0000 7f4e76ffd640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:24.098 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e76ffd640 1 -- 172.21.15.19:0/182689881 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80004d70 con 0x7f4e88139f40 2023-10-06T13:53:24.098 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e76ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:24.098 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e76ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:24.098 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e76ffd640 10 monclient: dump: 2023-10-06T13:53:24.098 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:24.098 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.098 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.098 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.098 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:24.099 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e76ffd640 10 monclient: _finish_auth 0 2023-10-06T13:53:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e76ffd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e76ffd640 1 -- 172.21.15.19:0/182689881 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e80007070 con 0x7f4e88139f40 2023-10-06T13:53:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e76ffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e76ffd640 1 -- 172.21.15.19:0/182689881 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80007420 con 0x7f4e88139f40 2023-10-06T13:53:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e76ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e76ffd640 10 monclient: dump: 2023-10-06T13:53:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:24.100 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e8cfc4640 5 monclient: authenticate success, global_id 4310 2023-10-06T13:53:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e8cfc4640 10 monclient: _renew_subs 2023-10-06T13:53:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e84d99640 4 set_mon_vals no callback set 2023-10-06T13:53:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e8cfc4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/182689881 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e880ac580 con 0x7f4e88139f40 2023-10-06T13:53:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e8cfc4640 10 client.4310.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e8cfc4640 10 monclient: _renew_subs 2023-10-06T13:53:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e8cfc4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/182689881 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e880aca40 con 0x7f4e88139f40 2023-10-06T13:53:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.096+0000 7f4e8cfc4640 1 librados: init done 2023-10-06T13:53:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.097+0000 7f4e8cfc4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.097+0000 7f4e8cfc4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.097+0000 7f4e8cfc4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.097+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/182689881 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8813aae0 con 0x7f4e88139f40 2023-10-06T13:53:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.098+0000 7fe3c1ffb640 1 -- 172.21.15.19:0/1630430231 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe3c804e450 con 0x7fe3cc0b00c0 2023-10-06T13:53:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.099+0000 7fe3c1ffb640 1 --2- 172.21.15.19:0/1630430231 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe3b006c1c0 0x7fe3b006e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.099+0000 7fe3d0c3f640 1 --2- 172.21.15.19:0/1630430231 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe3b006c1c0 0x7fe3b006e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.099+0000 7fe3d0c3f640 10 monclient: get_auth_request con 0x7fe3b006c1c0 auth_method 0 2023-10-06T13:53:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.099+0000 7fe3c1ffb640 10 client.4390.objecter ms_handle_connect 0x7fe3b006c1c0 2023-10-06T13:53:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.099+0000 7fe3d0c3f640 1 --2- 172.21.15.19:0/1630430231 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe3b006c1c0 0x7fe3b006e670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc13ca40 tx=0x7fe3bc007a40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.101+0000 7fe3c1ffb640 1 -- 172.21.15.19:0/1630430231 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe3c807ad90 con 0x7fe3cc0b00c0 2023-10-06T13:53:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.101+0000 7fe3c1ffb640 10 client.4390.objecter ms_dispatch 0x7fe3cc000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.101+0000 7fe3c1ffb640 3 client.4390.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.101+0000 7fe3c1ffb640 3 client.4390.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.101+0000 7fe3c1ffb640 20 client.4390.objecter dump_active .. 0 homeless 2023-10-06T13:53:24.112 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.111+0000 7fe3c1ffb640 1 -- 172.21.15.19:0/1630430231 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7fe3c80769d0 con 0x7fe3cc0b00c0 2023-10-06T13:53:24.112 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.111+0000 7fe3c1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:24.113 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.111+0000 7fe3c1ffb640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:24.125 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.124+0000 7f4e76ffd640 1 -- 172.21.15.19:0/182689881 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f4e80007580 con 0x7f4e88139f40 2023-10-06T13:53:24.126 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.124+0000 7f4e76ffd640 1 --2- 172.21.15.19:0/182689881 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f4e6806c1f0 0x7f4e6806e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.126 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.124+0000 7f4e8659c640 1 --2- 172.21.15.19:0/182689881 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f4e6806c1f0 0x7f4e6806e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:24.126 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.124+0000 7f4e8659c640 10 monclient: get_auth_request con 0x7f4e6806c1f0 auth_method 0 2023-10-06T13:53:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.125+0000 7f4e76ffd640 10 client.4310.objecter ms_handle_connect 0x7f4e6806c1f0 2023-10-06T13:53:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.125+0000 7f4e8659c640 1 --2- 172.21.15.19:0/182689881 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f4e6806c1f0 0x7f4e6806e6a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4e78033000 tx=0x7f4e78009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.133+0000 7fb01dd49640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2023-10-06T13:53:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.133+0000 7fb01dd49640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2023-10-06T13:53:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.133+0000 7fb01dd49640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.133+0000 7fb01dd49640 1 -- 172.21.15.19:0/927599858 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb0180678e0 con 0x7fb0180b00c0 2023-10-06T13:53:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.136+0000 7f8c695bb640 10 monclient: get_monmap_and_config 2023-10-06T13:53:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.136+0000 7f8c695bb640 10 monclient: build_initial_monmap 2023-10-06T13:53:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.136+0000 7f8c695bb640 10 monclient: monmap: 2023-10-06T13:53:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:24.137471+0000 2023-10-06T13:53:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:24.137471+0000 2023-10-06T13:53:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:24.139 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 1 Processor -- start 2023-10-06T13:53:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 1 -- start start 2023-10-06T13:53:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 10 monclient: init 2023-10-06T13:53:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 10 monclient: _add_conns ranks=[4,1,3,2,0] 2023-10-06T13:53:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8c640b3990 0x7f8c640af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 10 monclient(hunting): picked mon.noname-c con 0x7f8c640b3990 addr v2:172.21.15.83:3300/0 2023-10-06T13:53:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8c640b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8c640b00c0 0x7f8c640b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 10 monclient(hunting): picked mon.noname-a con 0x7f8c640b00c0 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c640aebe0 con 0x7f8c640b3d60 2023-10-06T13:53:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c640b0a60 con 0x7f8c640b00c0 2023-10-06T13:53:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c64140dd0 con 0x7f8c640b3990 2023-10-06T13:53:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.137+0000 7f8c695bb640 10 monclient(hunting): authenticate will time out at 1332.324209s 2023-10-06T13:53:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.138+0000 7f8c637fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8c640b00c0 0x7f8c640b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.139+0000 7f8c637fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8c640b00c0 0x7f8c640b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39186/0 (socket says 172.21.15.19:39186) 2023-10-06T13:53:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.139+0000 7f8c637fe640 1 -- 172.21.15.19:0/3654829258 learned_addr learned my addr 172.21.15.19:0/3654829258 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.139+0000 7f8c637fe640 10 monclient(hunting): get_auth_request con 0x7f8c640b00c0 auth_method 0 2023-10-06T13:53:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.139+0000 7f8c637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.139+0000 7f8c637fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.139+0000 7f8c637fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.139+0000 7f8c637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.139+0000 7f8c637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.139+0000 7f8c637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.142+0000 7f8c637fe640 10 monclient(hunting): handle_auth_done global_id 4386 payload 274 2023-10-06T13:53:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.142+0000 7f8c637fe640 1 -- 172.21.15.19:0/3654829258 >> v2:172.21.15.83:3300/0 conn(0x7f8c640b3990 msgr2=0x7f8c640af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.142+0000 7f8c637fe640 1 --2- 172.21.15.19:0/3654829258 >> v2:172.21.15.83:3300/0 conn(0x7f8c640b3990 0x7f8c640af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.142+0000 7f8c637fe640 1 -- 172.21.15.19:0/3654829258 >> v1:172.21.15.83:6789/0 conn(0x7f8c640b3d60 legacy=0x7f8c64141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.142+0000 7f8c637fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.142+0000 7f8c637fe640 1 monclient: found mon.noname-a 2023-10-06T13:53:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.142+0000 7f8c637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:24.145 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.142+0000 7f8c637fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:24.145 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.142+0000 7f8c637fe640 1 -- 172.21.15.19:0/3654829258 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c64141070 con 0x7f8c640b00c0 2023-10-06T13:53:24.146 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.145+0000 7f8c637fe640 1 --2- 172.21.15.19:0/3654829258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c640b00c0 0x7f8c640b0520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8c64072330 tx=0x7f8c5c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472a42fab27edb81 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c61ffb640 1 -- 172.21.15.19:0/3654829258 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c02c9f0 con 0x7f8c640b00c0 2023-10-06T13:53:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c61ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:24.152 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c61ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:24.152 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c61ffb640 10 monclient: dump: 2023-10-06T13:53:24.152 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:24.152 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.152 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.152 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.152 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:24.152 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:24.154 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c61ffb640 10 monclient: _finish_auth 0 2023-10-06T13:53:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c61ffb640 10 monclient: _check_auth_tickets 2023-10-06T13:53:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c61ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c61ffb640 1 -- 172.21.15.19:0/3654829258 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5c02cb50 con 0x7f8c640b00c0 2023-10-06T13:53:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c695bb640 5 monclient: authenticate success, global_id 4386 2023-10-06T13:53:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c695bb640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T13:53:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c61ffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c695bb640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c61ffb640 1 -- 172.21.15.19:0/3654829258 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0343f0 con 0x7f8c640b00c0 2023-10-06T13:53:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c695bb640 4 set_mon_vals no callback set 2023-10-06T13:53:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c61ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c61ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.149+0000 7f8c61ffb640 10 monclient: dump: 2023-10-06T13:53:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:24.156 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:24.156 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:24.156 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.150+0000 7f8c695bb640 10 monclient: shutdown 2023-10-06T13:53:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.150+0000 7f8c695bb640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.150+0000 7f8c695bb640 1 -- 172.21.15.19:0/3654829258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c640b00c0 msgr2=0x7f8c640b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.150+0000 7f8c695bb640 1 --2- 172.21.15.19:0/3654829258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c640b00c0 0x7f8c640b0520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8c64072330 tx=0x7f8c5c02be50 comp rx=0 tx=0).stop 2023-10-06T13:53:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.150+0000 7f8c695bb640 1 -- 172.21.15.19:0/3654829258 shutdown_connections 2023-10-06T13:53:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.150+0000 7f8c695bb640 1 --2- 172.21.15.19:0/3654829258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c640b00c0 0x7f8c640b0520 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.150+0000 7f8c695bb640 1 --2- 172.21.15.19:0/3654829258 >> v2:172.21.15.83:3300/0 conn(0x7f8c640b3990 0x7f8c640af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.150+0000 7f8c695bb640 1 -- 172.21.15.19:0/3654829258 >> 172.21.15.19:0/3654829258 conn(0x7f8c640b6e00 msgr2=0x7f8c640b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.150+0000 7f8c695bb640 1 -- 172.21.15.19:0/3654829258 shutdown_connections 2023-10-06T13:53:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.150+0000 7f8c695bb640 1 -- 172.21.15.19:0/3654829258 wait complete. 2023-10-06T13:53:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.151+0000 7f8c695bb640 10 monclient: build_initial_monmap 2023-10-06T13:53:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.151+0000 7f8c695bb640 10 monclient: monmap: 2023-10-06T13:53:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:24.160 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.151+0000 7f8c695bb640 1 librados: starting msgr at 2023-10-06T13:53:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.151+0000 7f8c695bb640 1 librados: starting objecter 2023-10-06T13:53:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.151+0000 7f8c695bb640 1 Processor -- start 2023-10-06T13:53:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 1 -- start start 2023-10-06T13:53:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 1 librados: setting wanted keys 2023-10-06T13:53:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 1 librados: calling monclient init 2023-10-06T13:53:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 10 monclient: init 2023-10-06T13:53:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T13:53:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b00c0 0x7f8c64140000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 10 monclient(hunting): picked mon.noname-b con 0x7f8c640b00c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:24.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8c640b3990 0x7f8c64143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 10 monclient(hunting): picked mon.noname-c con 0x7f8c640b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:24.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c640b3d60 0x7f8c64136ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 10 monclient(hunting): picked mon.noname-a con 0x7f8c640b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:24.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c64144990 con 0x7f8c640b3d60 2023-10-06T13:53:24.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c640a8920 con 0x7f8c640b00c0 2023-10-06T13:53:24.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c64144470 con 0x7f8c640b3990 2023-10-06T13:53:24.162 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.152+0000 7f8c695bb640 10 monclient(hunting): authenticate will time out at 1332.339026s 2023-10-06T13:53:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.153+0000 7f8c637fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c640b3d60 0x7f8c64136ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.153+0000 7f8c637fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c640b3d60 0x7f8c64136ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39196/0 (socket says 172.21.15.19:39196) 2023-10-06T13:53:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.153+0000 7f8c637fe640 1 -- 172.21.15.19:0/2442523565 learned_addr learned my addr 172.21.15.19:0/2442523565 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.153+0000 7f8c637fe640 10 monclient(hunting): get_auth_request con 0x7f8c640b3d60 auth_method 0 2023-10-06T13:53:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.153+0000 7f8c637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.153+0000 7f8c637fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.153+0000 7f8c637fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.153+0000 7f8c637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.153+0000 7f8c637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.153+0000 7f8c637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.156+0000 7f8c637fe640 10 monclient(hunting): handle_auth_done global_id 4389 payload 824 2023-10-06T13:53:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.156+0000 7f8c637fe640 1 -- 172.21.15.19:0/2442523565 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8c640b3990 msgr2=0x7f8c64143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.156+0000 7f8c637fe640 1 --2- 172.21.15.19:0/2442523565 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8c640b3990 0x7f8c64143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.156+0000 7f8c637fe640 1 -- 172.21.15.19:0/2442523565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b00c0 msgr2=0x7f8c64140000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.156+0000 7f8c637fe640 1 --2- 172.21.15.19:0/2442523565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b00c0 0x7f8c64140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.156+0000 7f8c637fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.156+0000 7f8c637fe640 1 monclient: found mon.noname-a 2023-10-06T13:53:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.156+0000 7f8c637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.156+0000 7f8c637fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.156+0000 7f8c637fe640 1 -- 172.21.15.19:0/2442523565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c009b30 con 0x7f8c640b3d60 2023-10-06T13:53:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.159+0000 7f8c637fe640 1 --2- 172.21.15.19:0/2442523565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c640b3d60 0x7f8c64136ff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c036040 tx=0x7f8c5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.159+0000 7f8c4b7fe640 10 client.?.objecter ms_handle_connect 0x7f8c640b3d60 2023-10-06T13:53:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.159+0000 7f8c4b7fe640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c4b7fe640 1 -- 172.21.15.19:0/2442523565 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c03e450 con 0x7f8c640b3d60 2023-10-06T13:53:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c4b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c4b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c4b7fe640 10 monclient: dump: 2023-10-06T13:53:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:24.167 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c4b7fe640 10 monclient: _finish_auth 0 2023-10-06T13:53:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c4b7fe640 10 monclient: _check_auth_tickets 2023-10-06T13:53:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c4b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c4b7fe640 1 -- 172.21.15.19:0/2442523565 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5c03e5b0 con 0x7f8c640b3d60 2023-10-06T13:53:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c4b7fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c4b7fe640 1 -- 172.21.15.19:0/2442523565 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c03e960 con 0x7f8c640b3d60 2023-10-06T13:53:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c4b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c4b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c4b7fe640 10 monclient: dump: 2023-10-06T13:53:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:24.169 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:24.169 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:24.169 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.169 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c617fa640 4 set_mon_vals no callback set 2023-10-06T13:53:24.169 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c695bb640 5 monclient: authenticate success, global_id 4389 2023-10-06T13:53:24.169 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c695bb640 10 monclient: _renew_subs 2023-10-06T13:53:24.169 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c695bb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:24.169 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c695bb640 1 -- 172.21.15.19:0/2442523565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c64143400 con 0x7f8c640b3d60 2023-10-06T13:53:24.169 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c695bb640 10 client.4389.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:24.169 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c695bb640 10 monclient: _renew_subs 2023-10-06T13:53:24.169 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c695bb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:24.169 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c695bb640 1 -- 172.21.15.19:0/2442523565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c64137710 con 0x7f8c640b3d60 2023-10-06T13:53:24.170 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.163+0000 7f8c695bb640 1 librados: init done 2023-10-06T13:53:24.170 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.164+0000 7f8c695bb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:24.170 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.164+0000 7f8c695bb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:24.170 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.164+0000 7f8c695bb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:24.170 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.164+0000 7f8c695bb640 1 -- 172.21.15.19:0/2442523565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c641442e0 con 0x7f8c640b3d60 2023-10-06T13:53:24.184 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.183+0000 7f8c4b7fe640 1 -- 172.21.15.19:0/2442523565 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8c5c03eac0 con 0x7f8c640b3d60 2023-10-06T13:53:24.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.183+0000 7f8c4b7fe640 1 --2- 172.21.15.19:0/2442523565 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8c5006c210 0x7f8c5006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.184+0000 7f8c62ffd640 1 --2- 172.21.15.19:0/2442523565 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8c5006c210 0x7f8c5006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:24.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.184+0000 7f8c62ffd640 10 monclient: get_auth_request con 0x7f8c5006c210 auth_method 0 2023-10-06T13:53:24.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.184+0000 7f8c62ffd640 1 --2- 172.21.15.19:0/2442523565 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8c5006c210 0x7f8c5006e6c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009710 tx=0x7f8c54006c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:24.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.184+0000 7f8c4b7fe640 10 client.4389.objecter ms_handle_connect 0x7f8c5006c210 2023-10-06T13:53:24.298 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.296+0000 7fe3d36a4640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2023-10-06T13:53:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.296+0000 7fe3d36a4640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2023-10-06T13:53:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.296+0000 7fe3d36a4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:53:24.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.296+0000 7fe3d36a4640 1 -- 172.21.15.19:0/1630430231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe3cc0ae200 con 0x7fe3cc0b00c0 2023-10-06T13:53:24.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.299+0000 7fe3c1ffb640 1 -- 172.21.15.19:0/1630430231 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe3c8035090 con 0x7fe3cc0b00c0 2023-10-06T13:53:24.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.300+0000 7fe3c1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2023-10-06T13:53:24.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.300+0000 7fe3c1ffb640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:24.302 INFO:teuthology.orchestra.run.smithi019.stdout:60129542150 2023-10-06T13:53:24.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.301+0000 7fe3d36a4640 10 librados: watch_flush enter 2023-10-06T13:53:24.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.301+0000 7fe3d36a4640 10 librados: watch_flush exit 2023-10-06T13:53:24.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.301+0000 7fe3d36a4640 20 client.4390.objecter shutdown clearing up homeless session... 2023-10-06T13:53:24.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.301+0000 7fe3d36a4640 10 client.4390.objecter successfully canceled tick 2023-10-06T13:53:24.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 1 -- 172.21.15.19:0/1630430231 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe3b006c1c0 msgr2=0x7fe3b006e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 1 --2- 172.21.15.19:0/1630430231 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe3b006c1c0 0x7fe3b006e670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc13ca40 tx=0x7fe3bc007a40 comp rx=0 tx=0).stop 2023-10-06T13:53:24.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 10 monclient: shutdown 2023-10-06T13:53:24.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 1 -- 172.21.15.19:0/1630430231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3cc0b00c0 msgr2=0x7fe3cc13c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 1 --2- 172.21.15.19:0/1630430231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3cc0b00c0 0x7fe3cc13c2a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3c802bee0 tx=0x7fe3c80046c0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 1 -- 172.21.15.19:0/1630430231 shutdown_connections 2023-10-06T13:53:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 1 --2- 172.21.15.19:0/1630430231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc0b3d60 0x7fe3cc1420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 1 --2- 172.21.15.19:0/1630430231 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe3b006c1c0 0x7fe3b006e670 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 1 --2- 172.21.15.19:0/1630430231 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe3cc0b3990 0x7fe3cc143e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 1 --2- 172.21.15.19:0/1630430231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3cc0b00c0 0x7fe3cc13c2a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 1 -- 172.21.15.19:0/1630430231 >> 172.21.15.19:0/1630430231 conn(0x7fe3cc0b6e00 msgr2=0x7fe3cc0b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 1 -- 172.21.15.19:0/1630430231 shutdown_connections 2023-10-06T13:53:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 1 -- 172.21.15.19:0/1630430231 wait complete. 2023-10-06T13:53:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.302+0000 7fe3d36a4640 1 librados: shutdown 2023-10-06T13:53:24.314 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2023-10-06T13:53:24.314 DEBUG:teuthology.parallel:result is None 2023-10-06T13:53:24.387 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.385+0000 7f4e76ffd640 1 -- 172.21.15.19:0/182689881 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4e80057ca0 con 0x7f4e88139f40 2023-10-06T13:53:24.388 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.385+0000 7f4e76ffd640 10 client.4310.objecter ms_dispatch 0x7f4e88000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:24.388 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.385+0000 7f4e76ffd640 3 client.4310.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:24.388 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.385+0000 7f4e76ffd640 3 client.4310.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:24.388 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.385+0000 7f4e76ffd640 20 client.4310.objecter dump_active .. 0 homeless 2023-10-06T13:53:24.388 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.385+0000 7f4e76ffd640 1 -- 172.21.15.19:0/182689881 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f4e800538e0 con 0x7f4e88139f40 2023-10-06T13:53:24.388 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.386+0000 7f4e76ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:24.388 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.386+0000 7f4e76ffd640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:24.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.389+0000 7faffffff640 1 -- 172.21.15.19:0/927599858 <== mon.2 v2:172.21.15.19:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb008017090 con 0x7fb0180b00c0 2023-10-06T13:53:24.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.389+0000 7faffffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2023-10-06T13:53:24.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.389+0000 7faffffff640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:24.391 INFO:teuthology.orchestra.run.smithi019.stdout:60129542149 2023-10-06T13:53:24.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 10 librados: watch_flush enter 2023-10-06T13:53:24.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 10 librados: watch_flush exit 2023-10-06T13:53:24.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 20 client.4307.objecter shutdown clearing up homeless session... 2023-10-06T13:53:24.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 10 client.4307.objecter successfully canceled tick 2023-10-06T13:53:24.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 1 -- 172.21.15.19:0/927599858 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fb00406c1c0 msgr2=0x7fb00406e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 1 --2- 172.21.15.19:0/927599858 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fb00406c1c0 0x7fb00406e670 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb018135920 tx=0x7fb00c006bf0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 10 monclient: shutdown 2023-10-06T13:53:24.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:24.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 1 -- 172.21.15.19:0/927599858 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb0180b00c0 msgr2=0x7fb018138a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 1 --2- 172.21.15.19:0/927599858 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb0180b00c0 0x7fb018138a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c410 tx=0x7fb00800c8d0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 1 -- 172.21.15.19:0/927599858 shutdown_connections 2023-10-06T13:53:24.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 1 --2- 172.21.15.19:0/927599858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0180b3d60 0x7fb018133760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 1 --2- 172.21.15.19:0/927599858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0180b3990 0x7fb018137de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 1 --2- 172.21.15.19:0/927599858 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fb00406c1c0 0x7fb00406e670 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 1 --2- 172.21.15.19:0/927599858 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb0180b00c0 0x7fb018138a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 1 -- 172.21.15.19:0/927599858 >> 172.21.15.19:0/927599858 conn(0x7fb0180b6e00 msgr2=0x7fb0180b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:24.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 1 -- 172.21.15.19:0/927599858 shutdown_connections 2023-10-06T13:53:24.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 1 -- 172.21.15.19:0/927599858 wait complete. 2023-10-06T13:53:24.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.391+0000 7fb01dd49640 1 librados: shutdown 2023-10-06T13:53:24.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.6 2023-10-06T13:53:24.405 DEBUG:teuthology.parallel:result is None 2023-10-06T13:53:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.447+0000 7f8c4b7fe640 1 -- 172.21.15.19:0/2442523565 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8c5c0473b0 con 0x7f8c640b3d60 2023-10-06T13:53:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.448+0000 7f8c4b7fe640 10 client.4389.objecter ms_dispatch 0x7f8c64000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.448+0000 7f8c4b7fe640 3 client.4389.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.448+0000 7f8c4b7fe640 3 client.4389.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.448+0000 7f8c4b7fe640 20 client.4389.objecter dump_active .. 0 homeless 2023-10-06T13:53:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.448+0000 7f8c4b7fe640 1 -- 172.21.15.19:0/2442523565 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f8c5c077d20 con 0x7f8c640b3d60 2023-10-06T13:53:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.448+0000 7f8c4b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.448+0000 7f8c4b7fe640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.573+0000 7f4e8cfc4640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2023-10-06T13:53:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.573+0000 7f4e8cfc4640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2023-10-06T13:53:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.573+0000 7f4e8cfc4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.573+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/182689881 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4e880aad20 con 0x7f4e88139f40 2023-10-06T13:53:24.582 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.580+0000 7f4e76ffd640 1 -- 172.21.15.19:0/182689881 <== mon.2 v2:172.21.15.19:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4e880aad20 con 0x7f4e88139f40 2023-10-06T13:53:24.582 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.580+0000 7f4e76ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2023-10-06T13:53:24.582 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.580+0000 7f4e76ffd640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:24.582 INFO:teuthology.orchestra.run.smithi019.stdout:68719476743 2023-10-06T13:53:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 10 librados: watch_flush enter 2023-10-06T13:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 10 librados: watch_flush exit 2023-10-06T13:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 20 client.4310.objecter shutdown clearing up homeless session... 2023-10-06T13:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 10 client.4310.objecter successfully canceled tick 2023-10-06T13:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/182689881 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f4e6806c1f0 msgr2=0x7f4e6806e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 1 --2- 172.21.15.19:0/182689881 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f4e6806c1f0 0x7f4e6806e6a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4e78033000 tx=0x7f4e78009030 comp rx=0 tx=0).stop 2023-10-06T13:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 10 monclient: shutdown 2023-10-06T13:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/182689881 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e88139f40 msgr2=0x7f4e8813a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 1 --2- 172.21.15.19:0/182689881 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e88139f40 0x7f4e8813a3b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4e80000f80 tx=0x7f4e8000ca70 comp rx=0 tx=0).stop 2023-10-06T13:53:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/182689881 shutdown_connections 2023-10-06T13:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 1 --2- 172.21.15.19:0/182689881 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f4e6806c1f0 0x7f4e6806e6a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 1 --2- 172.21.15.19:0/182689881 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f4e88139f40 0x7f4e8813a3b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 1 --2- 172.21.15.19:0/182689881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e880aa530 0x7f4e88144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 1 --2- 172.21.15.19:0/182689881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e70003970 0x7f4e880a9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/182689881 >> 172.21.15.19:0/182689881 conn(0x7f4e880b6e00 msgr2=0x7f4e880b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/182689881 shutdown_connections 2023-10-06T13:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 1 -- 172.21.15.19:0/182689881 wait complete. 2023-10-06T13:53:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.583+0000 7f4e8cfc4640 1 librados: shutdown 2023-10-06T13:53:24.600 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476743 for osd.0 2023-10-06T13:53:24.600 DEBUG:teuthology.parallel:result is None 2023-10-06T13:53:24.654 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.652+0000 7f8c695bb640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2023-10-06T13:53:24.654 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.652+0000 7f8c695bb640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-10-06T13:53:24.654 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.652+0000 7f8c695bb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:24.654 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.652+0000 7f8c695bb640 1 -- 172.21.15.19:0/2442523565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8c640678e0 con 0x7f8c640b3d60 2023-10-06T13:53:24.658 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.656+0000 7f8c4b7fe640 1 -- 172.21.15.19:0/2442523565 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8c5c043090 con 0x7f8c640b3d60 2023-10-06T13:53:24.658 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.656+0000 7f8c4b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-10-06T13:53:24.658 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.656+0000 7f8c4b7fe640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:24.658 INFO:teuthology.orchestra.run.smithi019.stdout:64424509447 2023-10-06T13:53:24.660 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 10 librados: watch_flush enter 2023-10-06T13:53:24.660 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 10 librados: watch_flush exit 2023-10-06T13:53:24.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 20 client.4389.objecter shutdown clearing up homeless session... 2023-10-06T13:53:24.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 10 client.4389.objecter successfully canceled tick 2023-10-06T13:53:24.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 1 -- 172.21.15.19:0/2442523565 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8c5006c210 msgr2=0x7f8c5006e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 1 --2- 172.21.15.19:0/2442523565 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8c5006c210 0x7f8c5006e6c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009710 tx=0x7f8c54006c90 comp rx=0 tx=0).stop 2023-10-06T13:53:24.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 10 monclient: shutdown 2023-10-06T13:53:24.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:24.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 1 -- 172.21.15.19:0/2442523565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c640b3d60 msgr2=0x7f8c64136ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 1 --2- 172.21.15.19:0/2442523565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c640b3d60 0x7f8c64136ff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c036040 tx=0x7f8c5c004a40 comp rx=0 tx=0).stop 2023-10-06T13:53:24.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 1 -- 172.21.15.19:0/2442523565 shutdown_connections 2023-10-06T13:53:24.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 1 --2- 172.21.15.19:0/2442523565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c640b3d60 0x7f8c64136ff0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 1 --2- 172.21.15.19:0/2442523565 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8c5006c210 0x7f8c5006e6c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 1 --2- 172.21.15.19:0/2442523565 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8c640b3990 0x7f8c64143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.662 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 1 --2- 172.21.15.19:0/2442523565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b00c0 0x7f8c64140000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.662 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 1 -- 172.21.15.19:0/2442523565 >> 172.21.15.19:0/2442523565 conn(0x7f8c640b6e00 msgr2=0x7f8c64141500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:24.662 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 1 -- 172.21.15.19:0/2442523565 shutdown_connections 2023-10-06T13:53:24.662 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 1 -- 172.21.15.19:0/2442523565 wait complete. 2023-10-06T13:53:24.662 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.658+0000 7f8c695bb640 1 librados: shutdown 2023-10-06T13:53:24.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.4 2023-10-06T13:53:24.671 DEBUG:teuthology.parallel:result is None 2023-10-06T13:53:24.671 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T13:53:24.671 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T13:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.775+0000 7fa4edba3640 10 monclient: get_monmap_and_config 2023-10-06T13:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.775+0000 7fa4edba3640 10 monclient: build_initial_monmap 2023-10-06T13:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.775+0000 7fa4edba3640 10 monclient: monmap: 2023-10-06T13:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:24.776446+0000 2023-10-06T13:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:24.776446+0000 2023-10-06T13:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:24.778 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.778 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:24.778 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:24.778 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:24.778 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:24.778 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:24.778 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.780 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 1 Processor -- start 2023-10-06T13:53:24.780 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 1 -- start start 2023-10-06T13:53:24.780 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 10 monclient: init 2023-10-06T13:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2023-10-06T13:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa4e806b6d0 0x7fa4e8144200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 10 monclient(hunting): picked mon.noname-a con 0x7fa4e806b6d0 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa4e8144740 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa4e813e630 0x7fa4e814b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 10 monclient(hunting): picked mon.noname-b con 0x7fa4e813e630 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e806baa0 con 0x7fa4e8144740 2023-10-06T13:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e80b2530 con 0x7fa4e806b6d0 2023-10-06T13:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e80b2810 con 0x7fa4e813e630 2023-10-06T13:53:24.782 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:24.782 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.776+0000 7fa4edba3640 10 monclient(hunting): authenticate will time out at 1332.963209s 2023-10-06T13:53:24.782 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.777+0000 7fa4e7fff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa4e813e630 0x7fa4e814b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:24.782 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.777+0000 7fa4e77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa4e806b6d0 0x7fa4e8144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:24.782 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.777+0000 7fa4e6ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa4e8144740 0x7fa4e813df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33034/0 (socket says 172.21.15.19:33034) 2023-10-06T13:53:24.782 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.777+0000 7fa4e6ffd640 1 -- 172.21.15.19:0/3988878945 learned_addr learned my addr 172.21.15.19:0/3988878945 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:24.782 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.777+0000 7fa4e7fff640 10 monclient(hunting): get_auth_request con 0x7fa4e813e630 auth_method 0 2023-10-06T13:53:24.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.777+0000 7fa4e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:24.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.777+0000 7fa4e7fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:24.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.777+0000 7fa4e7fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:24.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.778+0000 7fa4e77fe640 10 monclient(hunting): get_auth_request con 0x7fa4e806b6d0 auth_method 0 2023-10-06T13:53:24.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.778+0000 7fa4e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:24.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.778+0000 7fa4e77fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:24.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.778+0000 7fa4e77fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:24.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.778+0000 7fa4e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:24.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.778+0000 7fa4e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:24.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.778+0000 7fa4e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:24.783 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.779+0000 7fa4e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:24.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.779+0000 7fa4e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:24.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.779+0000 7fa4e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:24.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.780+0000 7fa4e7fff640 10 monclient(hunting): handle_auth_done global_id 4313 payload 274 2023-10-06T13:53:24.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.780+0000 7fa4e7fff640 1 -- 172.21.15.19:0/3988878945 >> v2:172.21.15.19:3300/0 conn(0x7fa4e806b6d0 msgr2=0x7fa4e8144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:24.784 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.780+0000 7fa4e7fff640 1 --2- 172.21.15.19:0/3988878945 >> v2:172.21.15.19:3300/0 conn(0x7fa4e806b6d0 0x7fa4e8144200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.780+0000 7fa4e7fff640 1 -- 172.21.15.19:0/3988878945 >> v1:172.21.15.19:6789/0 conn(0x7fa4e8144740 legacy=0x7fa4e813df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.780+0000 7fa4e7fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.780+0000 7fa4e7fff640 1 monclient: found mon.noname-b 2023-10-06T13:53:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.780+0000 7fa4e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.780+0000 7fa4e7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T13:53:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.780+0000 7fa4e7fff640 1 -- 172.21.15.19:0/3988878945 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e80aeff0 con 0x7fa4e813e630 2023-10-06T13:53:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.781+0000 7fa4e7fff640 1 --2- 172.21.15.19:0/3988878945 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa4e813e630 0x7fa4e814b530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009d90 tx=0x7fa4e002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1844b2dc2b62e3c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:24.787 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.785+0000 7fa4e67fc640 1 -- 172.21.15.19:0/3988878945 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e002cc50 con 0x7fa4e813e630 2023-10-06T13:53:24.787 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.785+0000 7fa4e67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:24.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4e67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:24.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4e67fc640 10 monclient: dump: 2023-10-06T13:53:24.788 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:24.788 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.788 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.788 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.788 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:24.788 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.788 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:24.788 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:24.789 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4e67fc640 10 monclient: _finish_auth 0 2023-10-06T13:53:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4e67fc640 10 monclient: _check_auth_tickets 2023-10-06T13:53:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4e67fc640 1 -- 172.21.15.19:0/3988878945 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e002cdb0 con 0x7fa4e813e630 2023-10-06T13:53:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4e67fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4e67fc640 1 -- 172.21.15.19:0/3988878945 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0034660 con 0x7fa4e813e630 2023-10-06T13:53:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4edba3640 5 monclient: authenticate success, global_id 4313 2023-10-06T13:53:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4edba3640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4edba3640 4 set_mon_vals no callback set 2023-10-06T13:53:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4e67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:24.790 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4e67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:24.790 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4e67fc640 10 monclient: dump: 2023-10-06T13:53:24.790 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:24.790 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.790 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.790 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.790 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:24.790 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.790 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:24.790 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:24.791 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:24.791 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4edba3640 10 monclient: shutdown 2023-10-06T13:53:24.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4edba3640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:24.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4edba3640 1 -- 172.21.15.19:0/3988878945 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa4e813e630 msgr2=0x7fa4e814b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4edba3640 1 --2- 172.21.15.19:0/3988878945 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa4e813e630 0x7fa4e814b530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009d90 tx=0x7fa4e002bea0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4edba3640 1 -- 172.21.15.19:0/3988878945 shutdown_connections 2023-10-06T13:53:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4edba3640 1 --2- 172.21.15.19:0/3988878945 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa4e813e630 0x7fa4e814b530 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4edba3640 1 --2- 172.21.15.19:0/3988878945 >> v2:172.21.15.19:3300/0 conn(0x7fa4e806b6d0 0x7fa4e8144200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4edba3640 1 -- 172.21.15.19:0/3988878945 >> 172.21.15.19:0/3988878945 conn(0x7fa4e80665b0 msgr2=0x7fa4e806ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4edba3640 1 -- 172.21.15.19:0/3988878945 shutdown_connections 2023-10-06T13:53:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.786+0000 7fa4edba3640 1 -- 172.21.15.19:0/3988878945 wait complete. 2023-10-06T13:53:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.787+0000 7fa4edba3640 10 monclient: build_initial_monmap 2023-10-06T13:53:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.787+0000 7fa4edba3640 10 monclient: monmap: 2023-10-06T13:53:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:24.796 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:24.796 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:24.796 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.796 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:24.796 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:24.796 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:24.796 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.787+0000 7fa4edba3640 1 librados: starting msgr at 2023-10-06T13:53:24.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.787+0000 7fa4edba3640 1 librados: starting objecter 2023-10-06T13:53:24.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.787+0000 7fa4edba3640 1 Processor -- start 2023-10-06T13:53:24.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 1 -- start start 2023-10-06T13:53:24.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 1 librados: setting wanted keys 2023-10-06T13:53:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 1 librados: calling monclient init 2023-10-06T13:53:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 10 monclient: init 2023-10-06T13:53:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T13:53:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e806b6d0 0x7fa4e81d9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 10 monclient(hunting): picked mon.noname-a con 0x7fa4e806b6d0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4e813e630 0x7fa4e80ac790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 10 monclient(hunting): picked mon.noname-b con 0x7fa4e813e630 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa4e8144740 0x7fa4e80accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 10 monclient(hunting): picked mon.noname-c con 0x7fa4e8144740 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81d3410 con 0x7fa4e806b6d0 2023-10-06T13:53:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81d8ef0 con 0x7fa4e813e630 2023-10-06T13:53:24.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:24.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e80ab210 con 0x7fa4e8144740 2023-10-06T13:53:24.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:24.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.788+0000 7fa4edba3640 10 monclient(hunting): authenticate will time out at 1332.975052s 2023-10-06T13:53:24.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e806b6d0 0x7fa4e81d9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:24.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e7fff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa4e8144740 0x7fa4e80accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:24.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e806b6d0 0x7fa4e81d9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39602/0 (socket says 172.21.15.19:39602) 2023-10-06T13:53:24.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e77fe640 1 -- 172.21.15.19:0/213952497 learned_addr learned my addr 172.21.15.19:0/213952497 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:24.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e77fe640 10 monclient(hunting): get_auth_request con 0x7fa4e806b6d0 auth_method 0 2023-10-06T13:53:24.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:24.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e77fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:24.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e77fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:24.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e6ffd640 1 --2- 172.21.15.19:0/213952497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4e813e630 0x7fa4e80ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:24.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e7fff640 10 monclient(hunting): get_auth_request con 0x7fa4e8144740 auth_method 0 2023-10-06T13:53:24.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:24.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e7fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:24.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e7fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:24.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e6ffd640 10 monclient(hunting): get_auth_request con 0x7fa4e813e630 auth_method 0 2023-10-06T13:53:24.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:24.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e6ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:24.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e6ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:24.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:24.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.789+0000 7fa4e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.790+0000 7fa4e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.790+0000 7fa4e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.790+0000 7fa4e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.790+0000 7fa4e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.790+0000 7fa4e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.790+0000 7fa4e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.790+0000 7fa4e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.792+0000 7fa4e77fe640 10 monclient(hunting): handle_auth_done global_id 4398 payload 824 2023-10-06T13:53:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.792+0000 7fa4e77fe640 1 -- 172.21.15.19:0/213952497 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa4e8144740 msgr2=0x7fa4e80accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.792+0000 7fa4e77fe640 1 --2- 172.21.15.19:0/213952497 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa4e8144740 0x7fa4e80accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.792+0000 7fa4e77fe640 1 -- 172.21.15.19:0/213952497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4e813e630 msgr2=0x7fa4e80ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.792+0000 7fa4e77fe640 1 --2- 172.21.15.19:0/213952497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4e813e630 0x7fa4e80ac790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.792+0000 7fa4e77fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.792+0000 7fa4e77fe640 1 monclient: found mon.noname-a 2023-10-06T13:53:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.792+0000 7fa4e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.792+0000 7fa4e77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.792+0000 7fa4e77fe640 1 -- 172.21.15.19:0/213952497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e0009a70 con 0x7fa4e806b6d0 2023-10-06T13:53:24.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.795+0000 7fa4e77fe640 1 --2- 172.21.15.19:0/213952497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e806b6d0 0x7fa4e81d9310 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800e9b0 tx=0x7fa4d800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:24.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.795+0000 7fa4cffff640 10 client.?.objecter ms_handle_connect 0x7fa4e806b6d0 2023-10-06T13:53:24.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.795+0000 7fa4cffff640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:24.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4cffff640 1 -- 172.21.15.19:0/213952497 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8009670 con 0x7fa4e806b6d0 2023-10-06T13:53:24.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4cffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:24.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4cffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:24.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4cffff640 10 monclient: dump: 2023-10-06T13:53:24.804 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:24.804 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.804 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.804 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.804 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:24.804 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.804 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:24.804 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:24.804 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:24.804 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4cffff640 10 monclient: _finish_auth 0 2023-10-06T13:53:24.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4cffff640 10 monclient: _check_auth_tickets 2023-10-06T13:53:24.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:24.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4cffff640 1 -- 172.21.15.19:0/213952497 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d800c9a0 con 0x7fa4e806b6d0 2023-10-06T13:53:24.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4cffff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:24.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4cffff640 1 -- 172.21.15.19:0/213952497 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800cd50 con 0x7fa4e806b6d0 2023-10-06T13:53:24.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4cffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:24.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4cffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:24.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4cffff640 10 monclient: dump: 2023-10-06T13:53:24.805 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:24.805 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:24.805 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.805 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:24.805 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:24.806 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:24.806 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:24.806 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:24.806 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:24.806 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:24.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4edba3640 5 monclient: authenticate success, global_id 4398 2023-10-06T13:53:24.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4edba3640 10 monclient: _renew_subs 2023-10-06T13:53:24.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4edba3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:24.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4edba3640 1 -- 172.21.15.19:0/213952497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e80ad2d0 con 0x7fa4e806b6d0 2023-10-06T13:53:24.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4e5ffb640 4 set_mon_vals no callback set 2023-10-06T13:53:24.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4edba3640 10 client.4398.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:24.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4edba3640 10 monclient: _renew_subs 2023-10-06T13:53:24.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4edba3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:24.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4edba3640 1 -- 172.21.15.19:0/213952497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e80a9680 con 0x7fa4e806b6d0 2023-10-06T13:53:24.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.798+0000 7fa4edba3640 1 librados: init done 2023-10-06T13:53:24.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.799+0000 7fa4edba3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:24.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.799+0000 7fa4edba3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:24.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.799+0000 7fa4edba3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:24.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.799+0000 7fa4edba3640 1 -- 172.21.15.19:0/213952497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e804e000 con 0x7fa4e806b6d0 2023-10-06T13:53:24.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.818+0000 7fa4cffff640 1 -- 172.21.15.19:0/213952497 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa4d8013470 con 0x7fa4e806b6d0 2023-10-06T13:53:24.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.818+0000 7fa4cffff640 1 --2- 172.21.15.19:0/213952497 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa4d406c210 0x7fa4d406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:24.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.819+0000 7fa4e6ffd640 1 --2- 172.21.15.19:0/213952497 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa4d406c210 0x7fa4d406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:24.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.819+0000 7fa4e6ffd640 10 monclient: get_auth_request con 0x7fa4d406c210 auth_method 0 2023-10-06T13:53:24.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.819+0000 7fa4cffff640 10 client.4398.objecter ms_handle_connect 0x7fa4d406c210 2023-10-06T13:53:24.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:24.819+0000 7fa4e6ffd640 1 --2- 172.21.15.19:0/213952497 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa4d406c210 0x7fa4d406e6c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa4e81443f0 tx=0x7fa4dc007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.074+0000 7fa4cffff640 1 -- 172.21.15.19:0/213952497 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa4d8058900 con 0x7fa4e806b6d0 2023-10-06T13:53:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.074+0000 7fa4cffff640 10 client.4398.objecter ms_dispatch 0x7fa4e8000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.074+0000 7fa4cffff640 3 client.4398.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.074+0000 7fa4cffff640 3 client.4398.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.074+0000 7fa4cffff640 20 client.4398.objecter dump_active .. 0 homeless 2023-10-06T13:53:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.074+0000 7fa4cffff640 1 -- 172.21.15.19:0/213952497 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7fa4d8054c40 con 0x7fa4e806b6d0 2023-10-06T13:53:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.074+0000 7fa4cffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.074+0000 7fa4cffff640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:25.267 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.266+0000 7fa4edba3640 1 -- 172.21.15.19:0/213952497 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa4e8144b60 con 0x7fa4d406c210 2023-10-06T13:53:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.268+0000 7fa4cffff640 1 -- 172.21.15.19:0/213952497 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47952 (secure 0 0 0) 0x7fa4dc0013c0 con 0x7fa4d406c210 2023-10-06T13:53:25.270 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:53:25.271 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2023-10-06T13:53:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 10 librados: watch_flush enter 2023-10-06T13:53:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 10 librados: watch_flush exit 2023-10-06T13:53:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 20 client.4398.objecter shutdown clearing up homeless session... 2023-10-06T13:53:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 10 client.4398.objecter successfully canceled tick 2023-10-06T13:53:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 1 -- 172.21.15.19:0/213952497 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa4d406c210 msgr2=0x7fa4d406e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 1 --2- 172.21.15.19:0/213952497 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa4d406c210 0x7fa4d406e6c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa4e81443f0 tx=0x7fa4dc007e80 comp rx=0 tx=0).stop 2023-10-06T13:53:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 10 monclient: shutdown 2023-10-06T13:53:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 1 -- 172.21.15.19:0/213952497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e806b6d0 msgr2=0x7fa4e81d9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 1 --2- 172.21.15.19:0/213952497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e806b6d0 0x7fa4e81d9310 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800e9b0 tx=0x7fa4d800ee70 comp rx=0 tx=0).stop 2023-10-06T13:53:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 1 -- 172.21.15.19:0/213952497 shutdown_connections 2023-10-06T13:53:25.274 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 1 --2- 172.21.15.19:0/213952497 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fa4d406c210 0x7fa4d406e6c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:25.274 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 1 --2- 172.21.15.19:0/213952497 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fa4e8144740 0x7fa4e80accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:25.274 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 1 --2- 172.21.15.19:0/213952497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4e813e630 0x7fa4e80ac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:25.274 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 1 --2- 172.21.15.19:0/213952497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e806b6d0 0x7fa4e81d9310 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:25.274 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.270+0000 7fa4edba3640 1 -- 172.21.15.19:0/213952497 >> 172.21.15.19:0/213952497 conn(0x7fa4e80665b0 msgr2=0x7fa4e813bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:25.274 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.271+0000 7fa4edba3640 1 -- 172.21.15.19:0/213952497 shutdown_connections 2023-10-06T13:53:25.274 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.271+0000 7fa4edba3640 1 -- 172.21.15.19:0/213952497 wait complete. 2023-10-06T13:53:25.274 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.271+0000 7fa4edba3640 1 librados: shutdown 2023-10-06T13:53:25.283 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2023-10-06T13:53:25.252085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218380,"kb_used_data":2576,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756340,"statfs":{"total":773094113280,"available":772870492160,"internally_reserved":0,"allocated":2637824,"data_stored":1435464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001549"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:18.113871+0000","last_change":"2023-10-06T13:53:17.220272+0000","last_active":"2023-10-06T13:53:18.113871+0000","last_peered":"2023-10-06T13:53:18.113871+0000","last_clean":"2023-10-06T13:53:18.113871+0000","last_became_active":"2023-10-06T13:53:17.214058+0000","last_became_peered":"2023-10-06T13:53:17.214058+0000","last_unstale":"2023-10-06T13:53:18.113871+0000","last_undegraded":"2023-10-06T13:53:18.113871+0000","last_fullsized":"2023-10-06T13:53:18.113871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:47.995210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:18.146928+0000","last_change":"2023-10-06T13:53:17.273582+0000","last_active":"2023-10-06T13:53:18.146928+0000","last_peered":"2023-10-06T13:53:18.146928+0000","last_clean":"2023-10-06T13:53:18.146928+0000","last_became_active":"2023-10-06T13:53:17.239679+0000","last_became_peered":"2023-10-06T13:53:17.239679+0000","last_unstale":"2023-10-06T13:53:18.146928+0000","last_undegraded":"2023-10-06T13:53:18.146928+0000","last_fullsized":"2023-10-06T13:53:18.146928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:23:07.122309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:18.113270+0000","last_change":"2023-10-06T13:53:16.872265+0000","last_active":"2023-10-06T13:53:18.113270+0000","last_peered":"2023-10-06T13:53:18.113270+0000","last_clean":"2023-10-06T13:53:18.113270+0000","last_became_active":"2023-10-06T13:53:16.861677+0000","last_became_peered":"2023-10-06T13:53:16.861677+0000","last_unstale":"2023-10-06T13:53:18.113270+0000","last_undegraded":"2023-10-06T13:53:18.113270+0000","last_fullsized":"2023-10-06T13:53:18.113270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:13:14.884882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:18.523446+0000","last_change":"2023-10-06T13:53:17.046864+0000","last_active":"2023-10-06T13:53:18.523446+0000","last_peered":"2023-10-06T13:53:18.523446+0000","last_clean":"2023-10-06T13:53:18.523446+0000","last_became_active":"2023-10-06T13:53:17.019550+0000","last_became_peered":"2023-10-06T13:53:17.019550+0000","last_unstale":"2023-10-06T13:53:18.523446+0000","last_undegraded":"2023-10-06T13:53:18.523446+0000","last_fullsized":"2023-10-06T13:53:18.523446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:55.095449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:18.111316+0000","last_change":"2023-10-06T13:53:17.012241+0000","last_active":"2023-10-06T13:53:18.111316+0000","last_peered":"2023-10-06T13:53:18.111316+0000","last_clean":"2023-10-06T13:53:18.111316+0000","last_became_active":"2023-10-06T13:53:17.002905+0000","last_became_peered":"2023-10-06T13:53:17.002905+0000","last_unstale":"2023-10-06T13:53:18.111316+0000","last_undegraded":"2023-10-06T13:53:18.111316+0000","last_fullsized":"2023-10-06T13:53:18.111316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:34:43.232087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:19.517394+0000","last_change":"2023-10-06T13:53:16.707767+0000","last_active":"2023-10-06T13:53:19.517394+0000","last_peered":"2023-10-06T13:53:19.517394+0000","last_clean":"2023-10-06T13:53:19.517394+0000","last_became_active":"2023-10-06T13:53:16.679265+0000","last_became_peered":"2023-10-06T13:53:16.679265+0000","last_unstale":"2023-10-06T13:53:19.517394+0000","last_undegraded":"2023-10-06T13:53:19.517394+0000","last_fullsized":"2023-10-06T13:53:19.517394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:50:43.442633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:18.106975+0000","last_change":"2023-10-06T13:53:16.801954+0000","last_active":"2023-10-06T13:53:18.106975+0000","last_peered":"2023-10-06T13:53:18.106975+0000","last_clean":"2023-10-06T13:53:18.106975+0000","last_became_active":"2023-10-06T13:53:16.767833+0000","last_became_peered":"2023-10-06T13:53:16.767833+0000","last_unstale":"2023-10-06T13:53:18.106975+0000","last_undegraded":"2023-10-06T13:53:18.106975+0000","last_fullsized":"2023-10-06T13:53:18.106975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:00.917851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:18.112594+0000","last_change":"2023-10-06T13:53:17.245905+0000","last_active":"2023-10-06T13:53:18.112594+0000","last_peered":"2023-10-06T13:53:18.112594+0000","last_clean":"2023-10-06T13:53:18.112594+0000","last_became_active":"2023-10-06T13:53:17.236625+0000","last_became_peered":"2023-10-06T13:53:17.236625+0000","last_unstale":"2023-10-06T13:53:18.112594+0000","last_undegraded":"2023-10-06T13:53:18.112594+0000","last_fullsized":"2023-10-06T13:53:18.112594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:13:27.212206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":118,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T13:53:22.562803+0000","last_change":"2023-10-06T13:53:11.205900+0000","last_active":"2023-10-06T13:53:22.562803+0000","last_peered":"2023-10-06T13:53:22.562803+0000","last_clean":"2023-10-06T13:53:22.562803+0000","last_became_active":"2023-10-06T13:53:11.174337+0000","last_became_peered":"2023-10-06T13:53:11.174337+0000","last_unstale":"2023-10-06T13:53:22.562803+0000","last_undegraded":"2023-10-06T13:53:22.562803+0000","last_fullsized":"2023-10-06T13:53:22.562803+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:08.156870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:08.156870+0000","last_clean_scrub_stamp":"2023-10-06T13:53:08.156870+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:49.733104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476744,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":630784,"data_stored":477813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.974}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":25.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.792999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.443000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.209}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.196999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.603999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476744,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":79973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.3190000000000008}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.073}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.5380000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.164}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.922000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.079999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.689}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":79973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7380000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.263999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.3559999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.890000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.923}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.034000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":79973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.298}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.823}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.542999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.6739999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.981999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.15}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.882}]}]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":79973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.866}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.587000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":50.244}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":24.611000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":30.600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":49.167999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.413}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":79973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.5049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.686}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.6050000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.529}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.837999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":630784,"data_stored":477813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8390000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.2479999999999993}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":29.684999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":30.298999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.779999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":31.431999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.252000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":79973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.541}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.005000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.395}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.603999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.032999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.33}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T13:53:25.285 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T13:53:25.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.388+0000 7f278bb7e640 10 monclient: get_monmap_and_config 2023-10-06T13:53:25.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.388+0000 7f278bb7e640 10 monclient: build_initial_monmap 2023-10-06T13:53:25.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.388+0000 7f278bb7e640 10 monclient: monmap: 2023-10-06T13:53:25.391 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:25.391 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:25.391 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:25.389165+0000 2023-10-06T13:53:25.391 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:25.389165+0000 2023-10-06T13:53:25.391 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:25.391 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:25.391 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:25.391 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:25.392 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:25.392 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:25.392 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:25.392 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:25.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.388+0000 7f278bb7e640 1 Processor -- start 2023-10-06T13:53:25.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.388+0000 7f278bb7e640 1 -- start start 2023-10-06T13:53:25.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.388+0000 7f278bb7e640 10 monclient: init 2023-10-06T13:53:25.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.389+0000 7f278bb7e640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:25.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.389+0000 7f278bb7e640 10 monclient: _add_conns ranks=[1,0,2,3,4] 2023-10-06T13:53:25.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.389+0000 7f278bb7e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f27840b3990 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:25.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.389+0000 7f278bb7e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f27840b3d60 addr v1:172.21.15.19:6789/0 2023-10-06T13:53:25.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.389+0000 7f278bb7e640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f27840b07d0 0x7f27840af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:25.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.389+0000 7f278bb7e640 10 monclient(hunting): picked mon.noname-b con 0x7f27840b07d0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:25.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.389+0000 7f278bb7e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27840aebe0 con 0x7f27840b3d60 2023-10-06T13:53:25.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.389+0000 7f278bb7e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2784141070 con 0x7f27840b3990 2023-10-06T13:53:25.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.389+0000 7f278bb7e640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:25.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.389+0000 7f278bb7e640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2784140dd0 con 0x7f27840b07d0 2023-10-06T13:53:25.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.389+0000 7f278bb7e640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:25.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.389+0000 7f278bb7e640 10 monclient(hunting): authenticate will time out at 1333.575905s 2023-10-06T13:53:25.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.390+0000 7f2789119640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f27840b3d60 0x7f27840b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33042/0 (socket says 172.21.15.19:33042) 2023-10-06T13:53:25.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.390+0000 7f2789119640 1 -- 172.21.15.19:0/402146042 learned_addr learned my addr 172.21.15.19:0/402146042 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:25.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.390+0000 7f278a11b640 1 --2- 172.21.15.19:0/402146042 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f27840b07d0 0x7f27840af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:25.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.390+0000 7f278a11b640 10 monclient(hunting): get_auth_request con 0x7f27840b07d0 auth_method 0 2023-10-06T13:53:25.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.390+0000 7f278a11b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:25.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.390+0000 7f278a11b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:25.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.390+0000 7f278a11b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:25.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.391+0000 7f278a11b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:25.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.391+0000 7f278a11b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:25.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.391+0000 7f278a11b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:25.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f2788918640 1 -- 172.21.15.19:0/402146042 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040111072 0 0) 0x7f2784141070 con 0x7f27840b3990 2023-10-06T13:53:25.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f2788918640 10 monclient(hunting): _negotiate 2023-10-06T13:53:25.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f2788918640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:25.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f2788918640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:25.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f2788918640 10 monclient(hunting): my global_id is 4396 2023-10-06T13:53:25.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f2788918640 1 -- 172.21.15.19:0/402146042 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2770003520 con 0x7f27840b3990 2023-10-06T13:53:25.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f2788918640 1 -- 172.21.15.19:0/402146042 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147335405 0 0) 0x7f27840aebe0 con 0x7f27840b3d60 2023-10-06T13:53:25.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f2788918640 10 monclient(hunting): _negotiate 2023-10-06T13:53:25.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f2788918640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:25.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f2788918640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:25.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f2788918640 10 monclient(hunting): my global_id is 4401 2023-10-06T13:53:25.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f2788918640 1 -- 172.21.15.19:0/402146042 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2784141070 con 0x7f27840b3d60 2023-10-06T13:53:25.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f278a11b640 10 monclient(hunting): handle_auth_done global_id 4319 payload 274 2023-10-06T13:53:25.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f278a11b640 1 -- 172.21.15.19:0/402146042 >> v1:172.21.15.83:6789/0 conn(0x7f27840b3990 legacy=0x7f2784141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:25.399 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f278a11b640 1 -- 172.21.15.19:0/402146042 >> v1:172.21.15.19:6789/0 conn(0x7f27840b3d60 legacy=0x7f27840b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:25.399 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f278a11b640 10 monclient: _finish_hunting 0 2023-10-06T13:53:25.399 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f278a11b640 1 monclient: found mon.noname-b 2023-10-06T13:53:25.399 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f278a11b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:25.399 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f278a11b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T13:53:25.399 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.392+0000 7f278a11b640 1 -- 172.21.15.19:0/402146042 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2784141c10 con 0x7f27840b07d0 2023-10-06T13:53:25.399 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.395+0000 7f278a11b640 1 --2- 172.21.15.19:0/402146042 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f27840b07d0 0x7f27840af810 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27800097d0 tx=0x7f278002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8c971039f12d3f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:25.400 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f2788918640 1 -- 172.21.15.19:0/402146042 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780031800 con 0x7f27840b07d0 2023-10-06T13:53:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f2788918640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f2788918640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:53:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f2788918640 10 monclient: dump: 2023-10-06T13:53:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:25.402 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f2788918640 10 monclient: _finish_auth 0 2023-10-06T13:53:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f2788918640 10 monclient: _check_auth_tickets 2023-10-06T13:53:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f2788918640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f2788918640 1 -- 172.21.15.19:0/402146042 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2780031960 con 0x7f27840b07d0 2023-10-06T13:53:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f2788918640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f2788918640 1 -- 172.21.15.19:0/402146042 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780031cf0 con 0x7f27840b07d0 2023-10-06T13:53:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f2788918640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f2788918640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f2788918640 10 monclient: dump: 2023-10-06T13:53:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:25.404 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f278bb7e640 5 monclient: authenticate success, global_id 4319 2023-10-06T13:53:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f278bb7e640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.398+0000 7f278bb7e640 4 set_mon_vals no callback set 2023-10-06T13:53:25.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.399+0000 7f278bb7e640 10 monclient: shutdown 2023-10-06T13:53:25.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.399+0000 7f278bb7e640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:25.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.399+0000 7f278bb7e640 1 -- 172.21.15.19:0/402146042 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f27840b07d0 msgr2=0x7f27840af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:25.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.399+0000 7f278bb7e640 1 --2- 172.21.15.19:0/402146042 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f27840b07d0 0x7f27840af810 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27800097d0 tx=0x7f278002b8a0 comp rx=0 tx=0).stop 2023-10-06T13:53:25.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.399+0000 7f278bb7e640 1 -- 172.21.15.19:0/402146042 shutdown_connections 2023-10-06T13:53:25.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.399+0000 7f278bb7e640 1 --2- 172.21.15.19:0/402146042 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f27840b07d0 0x7f27840af810 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.399+0000 7f278bb7e640 1 -- 172.21.15.19:0/402146042 >> 172.21.15.19:0/402146042 conn(0x7f27840b6e00 msgr2=0x7f27840b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.399+0000 7f278bb7e640 1 -- 172.21.15.19:0/402146042 shutdown_connections 2023-10-06T13:53:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.399+0000 7f278bb7e640 1 -- 172.21.15.19:0/402146042 wait complete. 2023-10-06T13:53:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.400+0000 7f278bb7e640 10 monclient: build_initial_monmap 2023-10-06T13:53:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.400+0000 7f278bb7e640 10 monclient: monmap: 2023-10-06T13:53:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:25.409 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.400+0000 7f278bb7e640 1 librados: starting msgr at 2023-10-06T13:53:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.400+0000 7f278bb7e640 1 librados: starting objecter 2023-10-06T13:53:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.400+0000 7f278bb7e640 1 Processor -- start 2023-10-06T13:53:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.400+0000 7f278bb7e640 1 -- start start 2023-10-06T13:53:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.400+0000 7f278bb7e640 1 librados: setting wanted keys 2023-10-06T13:53:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.400+0000 7f278bb7e640 1 librados: calling monclient init 2023-10-06T13:53:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.400+0000 7f278bb7e640 10 monclient: init 2023-10-06T13:53:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T13:53:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27840b07d0 0x7f278413d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 10 monclient(hunting): picked mon.noname-b con 0x7f27840b07d0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f27840b3990 0x7f27840a9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 10 monclient(hunting): picked mon.noname-c con 0x7f27840b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27840b3d60 0x7f27840ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 10 monclient(hunting): picked mon.noname-a con 0x7f27840b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27840a9930 con 0x7f27840b3d60 2023-10-06T13:53:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27840aa9f0 con 0x7f27840b07d0 2023-10-06T13:53:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27840aacd0 con 0x7f27840b3990 2023-10-06T13:53:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278bb7e640 10 monclient(hunting): authenticate will time out at 1333.587853s 2023-10-06T13:53:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.401+0000 7f278a11b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27840b3d60 0x7f27840ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f2789119640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f27840b3990 0x7f27840a9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f278a11b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27840b3d60 0x7f27840ab0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39604/0 (socket says 172.21.15.19:39604) 2023-10-06T13:53:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f278a11b640 1 -- 172.21.15.19:0/4035684519 learned_addr learned my addr 172.21.15.19:0/4035684519 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f278a11b640 10 monclient(hunting): get_auth_request con 0x7f27840b3d60 auth_method 0 2023-10-06T13:53:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f278a11b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f278a11b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f278a11b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f278991a640 1 --2- 172.21.15.19:0/4035684519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27840b07d0 0x7f278413d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f2789119640 10 monclient(hunting): get_auth_request con 0x7f27840b3990 auth_method 0 2023-10-06T13:53:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f2789119640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f2789119640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f2789119640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f278991a640 10 monclient(hunting): get_auth_request con 0x7f27840b07d0 auth_method 0 2023-10-06T13:53:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f278991a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f278991a640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f278991a640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f278a11b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f278a11b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:25.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f278a11b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:25.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f2789119640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:25.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f2789119640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:25.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.402+0000 7f2789119640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:25.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.403+0000 7f278991a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:25.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.403+0000 7f278991a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:25.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.403+0000 7f278991a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:25.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.405+0000 7f2789119640 10 monclient(hunting): handle_auth_done global_id 4322 payload 824 2023-10-06T13:53:25.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.405+0000 7f2789119640 1 -- 172.21.15.19:0/4035684519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27840b07d0 msgr2=0x7f278413d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:25.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.405+0000 7f2789119640 1 --2- 172.21.15.19:0/4035684519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27840b07d0 0x7f278413d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.405+0000 7f2789119640 1 -- 172.21.15.19:0/4035684519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27840b3d60 msgr2=0x7f27840ab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.405+0000 7f2789119640 1 --2- 172.21.15.19:0/4035684519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27840b3d60 0x7f27840ab0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.405+0000 7f2789119640 10 monclient: _finish_hunting 0 2023-10-06T13:53:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.405+0000 7f2789119640 1 monclient: found mon.noname-c 2023-10-06T13:53:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.405+0000 7f2789119640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.405+0000 7f2789119640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T13:53:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.405+0000 7f2789119640 1 -- 172.21.15.19:0/4035684519 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2780009400 con 0x7f27840b3990 2023-10-06T13:53:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.408+0000 7f2789119640 1 --2- 172.21.15.19:0/4035684519 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f27840b3990 0x7f27840a9130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f277400c390 tx=0x7f2774001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.408+0000 7f277a7fc640 10 client.?.objecter ms_handle_connect 0x7f27840b3990 2023-10-06T13:53:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.408+0000 7f277a7fc640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f277a7fc640 1 -- 172.21.15.19:0/4035684519 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2774005ce0 con 0x7f27840b3990 2023-10-06T13:53:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f277a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f277a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T13:53:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f277a7fc640 10 monclient: dump: 2023-10-06T13:53:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:25.417 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f277a7fc640 10 monclient: _finish_auth 0 2023-10-06T13:53:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f277a7fc640 10 monclient: _check_auth_tickets 2023-10-06T13:53:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f277a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f277a7fc640 1 -- 172.21.15.19:0/4035684519 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2774005e40 con 0x7f27840b3990 2023-10-06T13:53:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f278bb7e640 5 monclient: authenticate success, global_id 4322 2023-10-06T13:53:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f278bb7e640 10 monclient: _renew_subs 2023-10-06T13:53:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f278bb7e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f278bb7e640 1 -- 172.21.15.19:0/4035684519 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27840a8650 con 0x7f27840b3990 2023-10-06T13:53:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f277a7fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f277a7fc640 1 -- 172.21.15.19:0/4035684519 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27740072d0 con 0x7f27840b3990 2023-10-06T13:53:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f277a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f277a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:53:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f277a7fc640 10 monclient: dump: 2023-10-06T13:53:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:25.419 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f277bfff640 4 set_mon_vals no callback set 2023-10-06T13:53:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f278bb7e640 10 client.4322.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f278bb7e640 10 monclient: _renew_subs 2023-10-06T13:53:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f278bb7e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f278bb7e640 1 -- 172.21.15.19:0/4035684519 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27840ab630 con 0x7f27840b3990 2023-10-06T13:53:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.411+0000 7f278bb7e640 1 librados: init done 2023-10-06T13:53:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.412+0000 7f278bb7e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.412+0000 7f278bb7e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.412+0000 7f278bb7e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T13:53:25.420 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.412+0000 7f278bb7e640 1 -- 172.21.15.19:0/4035684519 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27840a9670 con 0x7f27840b3990 2023-10-06T13:53:25.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.432+0000 7f277a7fc640 1 -- 172.21.15.19:0/4035684519 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f27740074f0 con 0x7f27840b3990 2023-10-06T13:53:25.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.432+0000 7f277a7fc640 1 --2- 172.21.15.19:0/4035684519 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f276806c1c0 0x7f276806e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:25.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.432+0000 7f278991a640 1 --2- 172.21.15.19:0/4035684519 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f276806c1c0 0x7f276806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:25.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.432+0000 7f278991a640 10 monclient: get_auth_request con 0x7f276806c1c0 auth_method 0 2023-10-06T13:53:25.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.433+0000 7f277a7fc640 10 client.4322.objecter ms_handle_connect 0x7f276806c1c0 2023-10-06T13:53:25.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.433+0000 7f278991a640 1 --2- 172.21.15.19:0/4035684519 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f276806c1c0 0x7f276806e670 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f277c001560 tx=0x7f277c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:25.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.686+0000 7f277a7fc640 1 -- 172.21.15.19:0/4035684519 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f27740579a0 con 0x7f27840b3990 2023-10-06T13:53:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.686+0000 7f277a7fc640 10 client.4322.objecter ms_dispatch 0x7f2784000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.686+0000 7f277a7fc640 3 client.4322.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.686+0000 7f277a7fc640 3 client.4322.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.686+0000 7f277a7fc640 20 client.4322.objecter dump_active .. 0 homeless 2023-10-06T13:53:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.686+0000 7f277a7fc640 1 -- 172.21.15.19:0/4035684519 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f2774053ce0 con 0x7f27840b3990 2023-10-06T13:53:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.686+0000 7f277a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:25.690 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.686+0000 7f277a7fc640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:25.866 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.864+0000 7f278bb7e640 1 -- 172.21.15.19:0/4035684519 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f27840af3f0 con 0x7f276806c1c0 2023-10-06T13:53:25.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.867+0000 7f277a7fc640 1 -- 172.21.15.19:0/4035684519 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47952 (secure 0 0 0) 0x7f277c002080 con 0x7f276806c1c0 2023-10-06T13:53:25.869 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:53:25.869 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2023-10-06T13:53:25.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 10 librados: watch_flush enter 2023-10-06T13:53:25.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 10 librados: watch_flush exit 2023-10-06T13:53:25.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 20 client.4322.objecter shutdown clearing up homeless session... 2023-10-06T13:53:25.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 10 client.4322.objecter successfully canceled tick 2023-10-06T13:53:25.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 1 -- 172.21.15.19:0/4035684519 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f276806c1c0 msgr2=0x7f276806e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:25.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 1 --2- 172.21.15.19:0/4035684519 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f276806c1c0 0x7f276806e670 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f277c001560 tx=0x7f277c009040 comp rx=0 tx=0).stop 2023-10-06T13:53:25.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 10 monclient: shutdown 2023-10-06T13:53:25.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:25.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 1 -- 172.21.15.19:0/4035684519 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f27840b3990 msgr2=0x7f27840a9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:25.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 1 --2- 172.21.15.19:0/4035684519 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f27840b3990 0x7f27840a9130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f277400c390 tx=0x7f2774001e70 comp rx=0 tx=0).stop 2023-10-06T13:53:25.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 1 -- 172.21.15.19:0/4035684519 shutdown_connections 2023-10-06T13:53:25.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 1 --2- 172.21.15.19:0/4035684519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27840b3d60 0x7f27840ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:25.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 1 --2- 172.21.15.19:0/4035684519 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f276806c1c0 0x7f276806e670 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:25.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 1 --2- 172.21.15.19:0/4035684519 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f27840b3990 0x7f27840a9130 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:25.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 1 --2- 172.21.15.19:0/4035684519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27840b07d0 0x7f278413d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:25.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 1 -- 172.21.15.19:0/4035684519 >> 172.21.15.19:0/4035684519 conn(0x7f27840b6e00 msgr2=0x7f27840b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:25.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 1 -- 172.21.15.19:0/4035684519 shutdown_connections 2023-10-06T13:53:25.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 1 -- 172.21.15.19:0/4035684519 wait complete. 2023-10-06T13:53:25.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.869+0000 7f278bb7e640 1 librados: shutdown 2023-10-06T13:53:25.881 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2023-10-06T13:53:25.252085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218380,"kb_used_data":2576,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756340,"statfs":{"total":773094113280,"available":772870492160,"internally_reserved":0,"allocated":2637824,"data_stored":1435464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001549"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:18.113871+0000","last_change":"2023-10-06T13:53:17.220272+0000","last_active":"2023-10-06T13:53:18.113871+0000","last_peered":"2023-10-06T13:53:18.113871+0000","last_clean":"2023-10-06T13:53:18.113871+0000","last_became_active":"2023-10-06T13:53:17.214058+0000","last_became_peered":"2023-10-06T13:53:17.214058+0000","last_unstale":"2023-10-06T13:53:18.113871+0000","last_undegraded":"2023-10-06T13:53:18.113871+0000","last_fullsized":"2023-10-06T13:53:18.113871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:47.995210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:18.146928+0000","last_change":"2023-10-06T13:53:17.273582+0000","last_active":"2023-10-06T13:53:18.146928+0000","last_peered":"2023-10-06T13:53:18.146928+0000","last_clean":"2023-10-06T13:53:18.146928+0000","last_became_active":"2023-10-06T13:53:17.239679+0000","last_became_peered":"2023-10-06T13:53:17.239679+0000","last_unstale":"2023-10-06T13:53:18.146928+0000","last_undegraded":"2023-10-06T13:53:18.146928+0000","last_fullsized":"2023-10-06T13:53:18.146928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:23:07.122309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:18.113270+0000","last_change":"2023-10-06T13:53:16.872265+0000","last_active":"2023-10-06T13:53:18.113270+0000","last_peered":"2023-10-06T13:53:18.113270+0000","last_clean":"2023-10-06T13:53:18.113270+0000","last_became_active":"2023-10-06T13:53:16.861677+0000","last_became_peered":"2023-10-06T13:53:16.861677+0000","last_unstale":"2023-10-06T13:53:18.113270+0000","last_undegraded":"2023-10-06T13:53:18.113270+0000","last_fullsized":"2023-10-06T13:53:18.113270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:13:14.884882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:18.523446+0000","last_change":"2023-10-06T13:53:17.046864+0000","last_active":"2023-10-06T13:53:18.523446+0000","last_peered":"2023-10-06T13:53:18.523446+0000","last_clean":"2023-10-06T13:53:18.523446+0000","last_became_active":"2023-10-06T13:53:17.019550+0000","last_became_peered":"2023-10-06T13:53:17.019550+0000","last_unstale":"2023-10-06T13:53:18.523446+0000","last_undegraded":"2023-10-06T13:53:18.523446+0000","last_fullsized":"2023-10-06T13:53:18.523446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:55.095449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:18.111316+0000","last_change":"2023-10-06T13:53:17.012241+0000","last_active":"2023-10-06T13:53:18.111316+0000","last_peered":"2023-10-06T13:53:18.111316+0000","last_clean":"2023-10-06T13:53:18.111316+0000","last_became_active":"2023-10-06T13:53:17.002905+0000","last_became_peered":"2023-10-06T13:53:17.002905+0000","last_unstale":"2023-10-06T13:53:18.111316+0000","last_undegraded":"2023-10-06T13:53:18.111316+0000","last_fullsized":"2023-10-06T13:53:18.111316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:34:43.232087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:19.517394+0000","last_change":"2023-10-06T13:53:16.707767+0000","last_active":"2023-10-06T13:53:19.517394+0000","last_peered":"2023-10-06T13:53:19.517394+0000","last_clean":"2023-10-06T13:53:19.517394+0000","last_became_active":"2023-10-06T13:53:16.679265+0000","last_became_peered":"2023-10-06T13:53:16.679265+0000","last_unstale":"2023-10-06T13:53:19.517394+0000","last_undegraded":"2023-10-06T13:53:19.517394+0000","last_fullsized":"2023-10-06T13:53:19.517394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:50:43.442633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:18.106975+0000","last_change":"2023-10-06T13:53:16.801954+0000","last_active":"2023-10-06T13:53:18.106975+0000","last_peered":"2023-10-06T13:53:18.106975+0000","last_clean":"2023-10-06T13:53:18.106975+0000","last_became_active":"2023-10-06T13:53:16.767833+0000","last_became_peered":"2023-10-06T13:53:16.767833+0000","last_unstale":"2023-10-06T13:53:18.106975+0000","last_undegraded":"2023-10-06T13:53:18.106975+0000","last_fullsized":"2023-10-06T13:53:18.106975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:00.917851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T13:53:18.112594+0000","last_change":"2023-10-06T13:53:17.245905+0000","last_active":"2023-10-06T13:53:18.112594+0000","last_peered":"2023-10-06T13:53:18.112594+0000","last_clean":"2023-10-06T13:53:18.112594+0000","last_became_active":"2023-10-06T13:53:17.236625+0000","last_became_peered":"2023-10-06T13:53:17.236625+0000","last_unstale":"2023-10-06T13:53:18.112594+0000","last_undegraded":"2023-10-06T13:53:18.112594+0000","last_fullsized":"2023-10-06T13:53:18.112594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:13:27.212206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":118,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T13:53:22.562803+0000","last_change":"2023-10-06T13:53:11.205900+0000","last_active":"2023-10-06T13:53:22.562803+0000","last_peered":"2023-10-06T13:53:22.562803+0000","last_clean":"2023-10-06T13:53:22.562803+0000","last_became_active":"2023-10-06T13:53:11.174337+0000","last_became_peered":"2023-10-06T13:53:11.174337+0000","last_unstale":"2023-10-06T13:53:22.562803+0000","last_undegraded":"2023-10-06T13:53:22.562803+0000","last_fullsized":"2023-10-06T13:53:22.562803+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:08.156870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:08.156870+0000","last_clean_scrub_stamp":"2023-10-06T13:53:08.156870+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:49.733104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719476744,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":630784,"data_stored":477813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.974}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":25.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.792999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.443000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.209}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.196999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.603999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476744,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":79973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.3190000000000008}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.073}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.5380000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.164}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.922000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.079999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.689}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":79973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7380000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.263999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.3559999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.890000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.923}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.034000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":79973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.298}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.823}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.542999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.6739999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.981999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.15}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.882}]}]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":79973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.866}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.587000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":50.244}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":24.611000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":30.600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":49.167999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.413}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":79973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.5049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.686}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.6050000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.529}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.837999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":630784,"data_stored":477813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8390000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.2479999999999993}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":29.684999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":30.298999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.779999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":31.431999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.252000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":79973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.541}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.005000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.395}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.603999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.032999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.33}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T13:53:25.883 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T13:53:25.883 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T13:53:25.883 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T13:53:25.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T13:53:25.987 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.985+0000 7fbdae547640 10 monclient: get_monmap_and_config 2023-10-06T13:53:25.987 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.985+0000 7fbdae547640 10 monclient: build_initial_monmap 2023-10-06T13:53:25.988 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.985+0000 7fbdae547640 10 monclient: monmap: 2023-10-06T13:53:25.988 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:25.988 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:25.988 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:53:25.986854+0000 2023-10-06T13:53:25.988 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:53:25.986854+0000 2023-10-06T13:53:25.988 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:25.988 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:25.988 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:53:25.988 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:53:25.988 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:53:25.989 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:53:25.989 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:53:25.989 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.986+0000 7fbdae547640 1 Processor -- start 2023-10-06T13:53:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.986+0000 7fbdae547640 1 -- start start 2023-10-06T13:53:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.986+0000 7fbdae547640 10 monclient: init 2023-10-06T13:53:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbdae547640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbdae547640 10 monclient: _add_conns ranks=[1,3,2,4,0] 2023-10-06T13:53:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbdae547640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbda80662a0 addr v1:172.21.15.83:6789/0 2023-10-06T13:53:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbdae547640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbda81418a0 0x7fbda80af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbdae547640 10 monclient(hunting): picked mon.noname-a con 0x7fbda81418a0 addr v2:172.21.15.19:3300/0 2023-10-06T13:53:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbdae547640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbda80afc70 0x7fbda814b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbdae547640 10 monclient(hunting): picked mon.noname-b con 0x7fbda80afc70 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbdae547640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda8066670 con 0x7fbda80662a0 2023-10-06T13:53:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbdae547640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbdae547640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda80af030 con 0x7fbda81418a0 2023-10-06T13:53:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbdae547640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbdae547640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80af310 con 0x7fbda80afc70 2023-10-06T13:53:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbdae547640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbdae547640 10 monclient(hunting): authenticate will time out at 1334.173562s 2023-10-06T13:53:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.987+0000 7fbda77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbda81418a0 0x7fbda80af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.988+0000 7fbdacae4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbda80afc70 0x7fbda814b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.988+0000 7fbda7fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbda80662a0 0x7fbda8141190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:54326/0 (socket says 172.21.15.19:54326) 2023-10-06T13:53:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.988+0000 7fbda7fff640 1 -- 172.21.15.19:0/1202296880 learned_addr learned my addr 172.21.15.19:0/1202296880 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T13:53:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.988+0000 7fbda77fe640 10 monclient(hunting): get_auth_request con 0x7fbda81418a0 auth_method 0 2023-10-06T13:53:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.988+0000 7fbda77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.988+0000 7fbda77fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.988+0000 7fbda77fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.988+0000 7fbdacae4640 10 monclient(hunting): get_auth_request con 0x7fbda80afc70 auth_method 0 2023-10-06T13:53:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.988+0000 7fbdacae4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.988+0000 7fbdacae4640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.988+0000 7fbdacae4640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.988+0000 7fbda77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.988+0000 7fbda77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.988+0000 7fbda77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.989+0000 7fbdacae4640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.989+0000 7fbdacae4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.989+0000 7fbdacae4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.990+0000 7fbda77fe640 10 monclient(hunting): handle_auth_done global_id 4407 payload 274 2023-10-06T13:53:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.990+0000 7fbda77fe640 1 -- 172.21.15.19:0/1202296880 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbda80afc70 msgr2=0x7fbda814b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:53:25.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.990+0000 7fbda77fe640 1 --2- 172.21.15.19:0/1202296880 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbda80afc70 0x7fbda814b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:25.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.990+0000 7fbda77fe640 1 -- 172.21.15.19:0/1202296880 >> v1:172.21.15.83:6789/0 conn(0x7fbda80662a0 legacy=0x7fbda8141190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:25.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.990+0000 7fbda77fe640 10 monclient: _finish_hunting 0 2023-10-06T13:53:25.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.990+0000 7fbda77fe640 1 monclient: found mon.noname-a 2023-10-06T13:53:25.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.990+0000 7fbda77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:25.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.990+0000 7fbda77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:25.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.990+0000 7fbda77fe640 1 -- 172.21.15.19:0/1202296880 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda80b2740 con 0x7fbda81418a0 2023-10-06T13:53:25.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.992+0000 7fbda77fe640 1 --2- 172.21.15.19:0/1202296880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda81418a0 0x7fbda80af730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbd980097a0 tx=0x7fbd9802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b2fc5584e2f33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:25.997 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbda6ffd640 1 -- 172.21.15.19:0/1202296880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98031830 con 0x7fbda81418a0 2023-10-06T13:53:25.997 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbda6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:25.998 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbda6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:25.998 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbda6ffd640 10 monclient: dump: 2023-10-06T13:53:25.998 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:25.998 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:25.998 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:25.998 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:25.998 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:25.998 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:25.998 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:25.998 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:25.998 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:25.999 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbda6ffd640 10 monclient: _finish_auth 0 2023-10-06T13:53:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbda6ffd640 10 monclient: _check_auth_tickets 2023-10-06T13:53:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbda6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbda6ffd640 1 -- 172.21.15.19:0/1202296880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd98031990 con 0x7fbda81418a0 2023-10-06T13:53:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbda6ffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbdae547640 5 monclient: authenticate success, global_id 4407 2023-10-06T13:53:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbda6ffd640 1 -- 172.21.15.19:0/1202296880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98031d20 con 0x7fbda81418a0 2023-10-06T13:53:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbda6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbda6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbda6ffd640 10 monclient: dump: 2023-10-06T13:53:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:26.000 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbdae547640 10 monclient: get_monmap_and_config success 2023-10-06T13:53:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.995+0000 7fbdae547640 4 set_mon_vals no callback set 2023-10-06T13:53:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.996+0000 7fbdae547640 10 monclient: shutdown 2023-10-06T13:53:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.996+0000 7fbdae547640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.996+0000 7fbdae547640 1 -- 172.21.15.19:0/1202296880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda81418a0 msgr2=0x7fbda80af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.996+0000 7fbdae547640 1 --2- 172.21.15.19:0/1202296880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda81418a0 0x7fbda80af730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbd980097a0 tx=0x7fbd9802b8d0 comp rx=0 tx=0).stop 2023-10-06T13:53:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.996+0000 7fbdae547640 1 -- 172.21.15.19:0/1202296880 shutdown_connections 2023-10-06T13:53:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.996+0000 7fbdae547640 1 --2- 172.21.15.19:0/1202296880 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbda80afc70 0x7fbda814b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.996+0000 7fbdae547640 1 --2- 172.21.15.19:0/1202296880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda81418a0 0x7fbda80af730 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.996+0000 7fbdae547640 1 -- 172.21.15.19:0/1202296880 >> 172.21.15.19:0/1202296880 conn(0x7fbda806b9e0 msgr2=0x7fbda8065920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.996+0000 7fbdae547640 1 -- 172.21.15.19:0/1202296880 shutdown_connections 2023-10-06T13:53:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.996+0000 7fbdae547640 1 -- 172.21.15.19:0/1202296880 wait complete. 2023-10-06T13:53:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.997+0000 7fbdae547640 10 monclient: build_initial_monmap 2023-10-06T13:53:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.997+0000 7fbdae547640 10 monclient: monmap: 2023-10-06T13:53:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T13:53:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:53:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T13:53:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T13:53:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T13:53:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:53:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:53:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:53:26.005 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.997+0000 7fbdae547640 1 librados: starting msgr at 2023-10-06T13:53:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.997+0000 7fbdae547640 1 librados: starting objecter 2023-10-06T13:53:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.997+0000 7fbdae547640 1 Processor -- start 2023-10-06T13:53:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 1 -- start start 2023-10-06T13:53:26.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 1 librados: setting wanted keys 2023-10-06T13:53:26.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 1 librados: calling monclient init 2023-10-06T13:53:26.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 10 monclient: init 2023-10-06T13:53:26.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 10 monclient: _reopen_session rank -1 2023-10-06T13:53:26.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T13:53:26.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda80662a0 0x7fbda8143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:26.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 10 monclient(hunting): picked mon.noname-a con 0x7fbda80662a0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:53:26.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbda80afc70 0x7fbda80aa050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:26.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 10 monclient(hunting): picked mon.noname-c con 0x7fbda80afc70 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:53:26.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81418a0 0x7fbda81d8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:26.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 10 monclient(hunting): picked mon.noname-b con 0x7fbda81418a0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:53:26.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda813ad30 con 0x7fbda80662a0 2023-10-06T13:53:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80aa8e0 con 0x7fbda81418a0 2023-10-06T13:53:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 10 monclient(hunting): start opening mon connection 2023-10-06T13:53:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80aa590 con 0x7fbda80afc70 2023-10-06T13:53:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 10 monclient(hunting): _renew_subs 2023-10-06T13:53:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.998+0000 7fbdae547640 10 monclient(hunting): authenticate will time out at 1334.185098s 2023-10-06T13:53:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbda7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda80662a0 0x7fbda8143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbda77fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbda80afc70 0x7fbda80aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbda7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda80662a0 0x7fbda8143ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39610/0 (socket says 172.21.15.19:39610) 2023-10-06T13:53:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbda7fff640 1 -- 172.21.15.19:0/1879633510 learned_addr learned my addr 172.21.15.19:0/1879633510 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:53:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbda7fff640 10 monclient(hunting): get_auth_request con 0x7fbda80662a0 auth_method 0 2023-10-06T13:53:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbda7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbda7fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbda7fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbdacae4640 1 --2- 172.21.15.19:0/1879633510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81418a0 0x7fbda81d8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbda77fe640 10 monclient(hunting): get_auth_request con 0x7fbda80afc70 auth_method 0 2023-10-06T13:53:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbda77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbda77fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbda77fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbdacae4640 10 monclient(hunting): get_auth_request con 0x7fbda81418a0 auth_method 0 2023-10-06T13:53:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbdacae4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:53:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbdacae4640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:53:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:25.999+0000 7fbdacae4640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:53:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.000+0000 7fbda7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.000+0000 7fbda7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.000+0000 7fbda7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.000+0000 7fbda77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.000+0000 7fbda77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.000+0000 7fbda77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.000+0000 7fbdacae4640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:53:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.000+0000 7fbdacae4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:53:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.000+0000 7fbdacae4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:53:26.010 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.002+0000 7fbda7fff640 10 monclient(hunting): handle_auth_done global_id 4410 payload 824 2023-10-06T13:53:26.010 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.003+0000 7fbda7fff640 1 -- 172.21.15.19:0/1879633510 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbda80afc70 msgr2=0x7fbda80aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:26.010 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.003+0000 7fbda7fff640 1 --2- 172.21.15.19:0/1879633510 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbda80afc70 0x7fbda80aa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:26.010 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.003+0000 7fbda7fff640 1 -- 172.21.15.19:0/1879633510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81418a0 msgr2=0x7fbda81d8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:26.010 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.003+0000 7fbda7fff640 1 --2- 172.21.15.19:0/1879633510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81418a0 0x7fbda81d8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:26.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.003+0000 7fbda7fff640 10 monclient: _finish_hunting 0 2023-10-06T13:53:26.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.003+0000 7fbda7fff640 1 monclient: found mon.noname-a 2023-10-06T13:53:26.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.003+0000 7fbda7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:53:26.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.003+0000 7fbda7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:53:26.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.003+0000 7fbda7fff640 1 -- 172.21.15.19:0/1879633510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd98009400 con 0x7fbda80662a0 2023-10-06T13:53:26.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.005+0000 7fbda7fff640 1 --2- 172.21.15.19:0/1879633510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda80662a0 0x7fbda8143ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00c400 tx=0x7fbd9c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:26.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.005+0000 7fbda4ff9640 10 client.?.objecter ms_handle_connect 0x7fbda80662a0 2023-10-06T13:53:26.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.005+0000 7fbda4ff9640 10 client.?.objecter resend_mon_ops 2023-10-06T13:53:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbda4ff9640 1 -- 172.21.15.19:0/1879633510 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c005ce0 con 0x7fbda80662a0 2023-10-06T13:53:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbda4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbda4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:53:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbda4ff9640 10 monclient: dump: 2023-10-06T13:53:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:26.013 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbda4ff9640 10 monclient: _finish_auth 0 2023-10-06T13:53:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbda4ff9640 10 monclient: _check_auth_tickets 2023-10-06T13:53:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbda4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:53:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbda4ff9640 1 -- 172.21.15.19:0/1879633510 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c005e40 con 0x7fbda80662a0 2023-10-06T13:53:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbda4ff9640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:53:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbda4ff9640 1 -- 172.21.15.19:0/1879633510 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0072d0 con 0x7fbda80662a0 2023-10-06T13:53:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbdae547640 5 monclient: authenticate success, global_id 4410 2023-10-06T13:53:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbdae547640 10 monclient: _renew_subs 2023-10-06T13:53:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbdae547640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbdae547640 1 -- 172.21.15.19:0/1879633510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda81449f0 con 0x7fbda80662a0 2023-10-06T13:53:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbda4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:53:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbda4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:53:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbda4ff9640 10 monclient: dump: 2023-10-06T13:53:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T13:53:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:53:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:53:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T13:53:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T13:53:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:53:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:53:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:53:26.015 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T13:53:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbda67fc640 4 set_mon_vals no callback set 2023-10-06T13:53:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbdae547640 10 client.4410.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:53:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbdae547640 10 monclient: _renew_subs 2023-10-06T13:53:26.016 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbdae547640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:26.016 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbdae547640 1 -- 172.21.15.19:0/1879633510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda80aaff0 con 0x7fbda80662a0 2023-10-06T13:53:26.016 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.009+0000 7fbdae547640 1 librados: init done 2023-10-06T13:53:26.016 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.010+0000 7fbdae547640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:26.016 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.010+0000 7fbdae547640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:26.016 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.010+0000 7fbdae547640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:26.016 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.010+0000 7fbdae547640 1 -- 172.21.15.19:0/1879633510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda804e000 con 0x7fbda80662a0 2023-10-06T13:53:26.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.028+0000 7fbda4ff9640 1 -- 172.21.15.19:0/1879633510 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fbd9c0074f0 con 0x7fbda80662a0 2023-10-06T13:53:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.028+0000 7fbda4ff9640 1 --2- 172.21.15.19:0/1879633510 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbd9406c210 0x7fbd9406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:53:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.028+0000 7fbda77fe640 1 --2- 172.21.15.19:0/1879633510 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbd9406c210 0x7fbd9406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:53:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.028+0000 7fbda77fe640 10 monclient: get_auth_request con 0x7fbd9406c210 auth_method 0 2023-10-06T13:53:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.029+0000 7fbda4ff9640 10 client.4410.objecter ms_handle_connect 0x7fbd9406c210 2023-10-06T13:53:26.031 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.029+0000 7fbda77fe640 1 --2- 172.21.15.19:0/1879633510 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbd9406c210 0x7fbd9406e6c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbd98036000 tx=0x7fbd9803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:53:26.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.031+0000 7fbda4ff9640 1 -- 172.21.15.19:0/1879633510 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbd9c057830 con 0x7fbda80662a0 2023-10-06T13:53:26.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.031+0000 7fbda4ff9640 10 client.4410.objecter ms_dispatch 0x7fbda8000b90 osd_map(23..23 src has 1..23) v4 2023-10-06T13:53:26.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.031+0000 7fbda4ff9640 3 client.4410.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:53:26.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.031+0000 7fbda4ff9640 3 client.4410.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:53:26.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.031+0000 7fbda4ff9640 20 client.4410.objecter dump_active .. 0 homeless 2023-10-06T13:53:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.306+0000 7fbda4ff9640 1 -- 172.21.15.19:0/1879633510 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7fbd9c053b70 con 0x7fbda80662a0 2023-10-06T13:53:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.306+0000 7fbda4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T13:53:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.306+0000 7fbda4ff9640 10 monclient: _finish_command 1 = system:0 2023-10-06T13:53:26.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.521+0000 7fbdae547640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2023-10-06T13:53:26.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.521+0000 7fbdae547640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2023-10-06T13:53:26.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.521+0000 7fbdae547640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:53:26.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.521+0000 7fbdae547640 1 -- 172.21.15.19:0/1879633510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbda80aaff0 con 0x7fbda80662a0 2023-10-06T13:53:26.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.527+0000 7fbda4ff9640 1 -- 172.21.15.19:0/1879633510 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fbda80aaff0 con 0x7fbda80662a0 2023-10-06T13:53:26.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.527+0000 7fbda4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2023-10-06T13:53:26.529 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.527+0000 7fbda4ff9640 10 monclient: _finish_command 2 = system:0 2023-10-06T13:53:26.529 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T13:53:26.529 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T13:53:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 10 librados: watch_flush enter 2023-10-06T13:53:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 10 librados: watch_flush exit 2023-10-06T13:53:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 20 client.4410.objecter shutdown clearing up homeless session... 2023-10-06T13:53:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 10 client.4410.objecter successfully canceled tick 2023-10-06T13:53:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 1 -- 172.21.15.19:0/1879633510 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbd9406c210 msgr2=0x7fbd9406e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 1 --2- 172.21.15.19:0/1879633510 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbd9406c210 0x7fbd9406e6c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbd98036000 tx=0x7fbd9803c040 comp rx=0 tx=0).stop 2023-10-06T13:53:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 10 monclient: shutdown 2023-10-06T13:53:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:53:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 1 -- 172.21.15.19:0/1879633510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda80662a0 msgr2=0x7fbda8143ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:53:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 1 --2- 172.21.15.19:0/1879633510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda80662a0 0x7fbda8143ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00c400 tx=0x7fbd9c00c8c0 comp rx=0 tx=0).stop 2023-10-06T13:53:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 1 -- 172.21.15.19:0/1879633510 shutdown_connections 2023-10-06T13:53:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 1 --2- 172.21.15.19:0/1879633510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81418a0 0x7fbda81d8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 1 --2- 172.21.15.19:0/1879633510 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbd9406c210 0x7fbd9406e6c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 1 --2- 172.21.15.19:0/1879633510 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbda80afc70 0x7fbda80aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 1 --2- 172.21.15.19:0/1879633510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda80662a0 0x7fbda8143ea0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:53:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 1 -- 172.21.15.19:0/1879633510 >> 172.21.15.19:0/1879633510 conn(0x7fbda806b9e0 msgr2=0x7fbda8140f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:53:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 1 -- 172.21.15.19:0/1879633510 shutdown_connections 2023-10-06T13:53:26.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 1 -- 172.21.15.19:0/1879633510 wait complete. 2023-10-06T13:53:26.534 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T13:53:26.530+0000 7fbdae547640 1 librados: shutdown 2023-10-06T13:53:26.544 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T13:53:26.544 INFO:teuthology.run_tasks:Running task workunit... 2023-10-06T13:53:26.556 INFO:tasks.workunit:Pulling workunits from ref 4af54c4a1461e3792e9783e337bb62ce429214f3 2023-10-06T13:53:26.557 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-06T13:53:26.557 DEBUG:teuthology.orchestra.run.smithi019:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-06T13:53:26.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T13:53:26.572 INFO:teuthology.orchestra.run.smithi019.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T13:53:26.572 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -- /home/ubuntu/cephtest/mnt.0 2023-10-06T13:53:26.628 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2023-10-06T13:53:26.628 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2023-10-06T13:53:26.683 INFO:tasks.workunit:timeout=6h 2023-10-06T13:53:26.683 INFO:tasks.workunit:cleanup=True 2023-10-06T13:53:26.683 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 4af54c4a1461e3792e9783e337bb62ce429214f3 2023-10-06T13:53:26.738 INFO:tasks.workunit.client.0.smithi019.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-06T13:53:37.650 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:01:40.773 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:37.658 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:01:40.781 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:37.701 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:01:40.798 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:37.704 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:01:40.827 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:37.706 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:01:40.803 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:37.714 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:01:40.837 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:37.720 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:01:40.843 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:37.743 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:01:40.840 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:37.763 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:01:40.886 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:57.790 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:02:00.913 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:57.796 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:02:00.919 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:57.822 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:02:00.945 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:57.841 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:02:00.964 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:57.841 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:02:00.938 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:57.846 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:02:00.943 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:57.864 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:02:00.961 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:57.882 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:02:01.005 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:53:57.914 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:02:01.037 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:13.130 INFO:tasks.workunit.client.0.smithi019.stderr:Note: switching to '4af54c4a1461e3792e9783e337bb62ce429214f3'. 2023-10-06T13:54:13.130 INFO:tasks.workunit.client.0.smithi019.stderr: 2023-10-06T13:54:13.130 INFO:tasks.workunit.client.0.smithi019.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-06T13:54:13.130 INFO:tasks.workunit.client.0.smithi019.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-06T13:54:13.130 INFO:tasks.workunit.client.0.smithi019.stderr:state without impacting any branches by switching back to a branch. 2023-10-06T13:54:13.130 INFO:tasks.workunit.client.0.smithi019.stderr: 2023-10-06T13:54:13.130 INFO:tasks.workunit.client.0.smithi019.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-06T13:54:13.131 INFO:tasks.workunit.client.0.smithi019.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-06T13:54:13.131 INFO:tasks.workunit.client.0.smithi019.stderr: 2023-10-06T13:54:13.131 INFO:tasks.workunit.client.0.smithi019.stderr: git switch -c 2023-10-06T13:54:13.131 INFO:tasks.workunit.client.0.smithi019.stderr: 2023-10-06T13:54:13.131 INFO:tasks.workunit.client.0.smithi019.stderr:Or undo this operation with: 2023-10-06T13:54:13.131 INFO:tasks.workunit.client.0.smithi019.stderr: 2023-10-06T13:54:13.131 INFO:tasks.workunit.client.0.smithi019.stderr: git switch - 2023-10-06T13:54:13.131 INFO:tasks.workunit.client.0.smithi019.stderr: 2023-10-06T13:54:13.131 INFO:tasks.workunit.client.0.smithi019.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-06T13:54:13.131 INFO:tasks.workunit.client.0.smithi019.stderr: 2023-10-06T13:54:13.131 INFO:tasks.workunit.client.0.smithi019.stderr:HEAD is now at 4af54c4a146 Merge branch 'wip-63083-reef' of https://github.com/ljflores/ceph into wip-yuri7-testing-2023-10-04-1350-reef 2023-10-06T13:54:13.134 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-06T13:54:13.197 INFO:tasks.workunit.client.0.smithi019.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-06T13:54:13.199 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-06T13:54:13.199 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-06T13:54:13.844 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-06T13:54:13.932 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-06T13:54:13.974 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-06T13:54:13.976 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-06T13:54:13.976 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-06T13:54:14.018 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-06T13:54:14.021 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-10-06T13:54:14.021 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-06T13:54:14.076 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2023-10-06T13:54:14.077 INFO:tasks.workunit:Running workunit rados/test.sh... 2023-10-06T13:54:14.077 DEBUG:teuthology.orchestra.run.smithi019:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4af54c4a1461e3792e9783e337bb62ce429214f3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2023-10-06T13:54:14.137 INFO:tasks.workunit.client.0.smithi019.stderr:+ parallel=1 2023-10-06T13:54:14.137 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' '' = --serial ']' 2023-10-06T13:54:14.137 INFO:tasks.workunit.client.0.smithi019.stderr:+ color= 2023-10-06T13:54:14.138 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -t 1 ']' 2023-10-06T13:54:14.138 INFO:tasks.workunit.client.0.smithi019.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2023-10-06T13:54:14.138 INFO:tasks.workunit.client.0.smithi019.stderr:+ declare -A pids 2023-10-06T13:54:14.138 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.138 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.138 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_aio 2023-10-06T13:54:14.138 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_aio' 2023-10-06T13:54:14.138 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_aio 2023-10-06T13:54:14.138 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.139 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_aio 2023-10-06T13:54:14.139 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91621 2023-10-06T13:54:14.139 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_aio on pid 91621' 2023-10-06T13:54:14.139 INFO:tasks.workunit.client.0.smithi019.stdout:test api_aio on pid 91621 2023-10-06T13:54:14.142 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91621 2023-10-06T13:54:14.142 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.142 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.143 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2023-10-06T13:54:14.143 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_aio_pp 2023-10-06T13:54:14.143 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_aio_pp' 2023-10-06T13:54:14.143 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_aio_pp 2023-10-06T13:54:14.143 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.143 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.143 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.143 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.143 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.143 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.143 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.144 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.144 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.144 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.144 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.144 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:14.144 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.144 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:14.144 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:14.144 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.144 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:14.144 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.144 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:14.144 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.145 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.145 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.145 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:14.145 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.145 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:14.145 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.145 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_aio_pp 2023-10-06T13:54:14.145 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91628 2023-10-06T13:54:14.145 INFO:tasks.workunit.client.0.smithi019.stdout:test api_aio_pp on pid 91628 2023-10-06T13:54:14.149 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_aio_pp on pid 91628' 2023-10-06T13:54:14.149 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91628 2023-10-06T13:54:14.149 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.149 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.149 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2023-10-06T13:54:14.149 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_io 2023-10-06T13:54:14.150 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_io' 2023-10-06T13:54:14.150 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_io 2023-10-06T13:54:14.150 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.150 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.150 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.150 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.150 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.150 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.150 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.151 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.151 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.151 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.151 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:14.151 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:14.151 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.151 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:14.151 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.151 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:14.151 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:14.151 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.152 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:14.152 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.152 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:14.152 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:14.152 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.152 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:14.152 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.152 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:14.152 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.152 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.152 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:14.152 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:14.153 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.153 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:14.153 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.153 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:14.153 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.153 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:14.153 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.153 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.153 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:14.153 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:14.153 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:14.153 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.154 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:14.154 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.154 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:14.154 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.154 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_io 2023-10-06T13:54:14.154 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91637 2023-10-06T13:54:14.154 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_io on pid 91637' 2023-10-06T13:54:14.154 INFO:tasks.workunit.client.0.smithi019.stdout:test api_io on pid 91637 2023-10-06T13:54:14.158 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91637 2023-10-06T13:54:14.158 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.158 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.158 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2023-10-06T13:54:14.158 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_io_pp 2023-10-06T13:54:14.158 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_io_pp' 2023-10-06T13:54:14.158 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_io_pp 2023-10-06T13:54:14.158 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.159 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:14.159 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:14.159 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.159 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.159 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:14.159 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.159 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:14.159 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.159 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:14.159 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.159 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.159 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.159 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:14.160 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.160 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:14.160 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.160 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:14.160 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.160 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:14.160 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:14.160 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:14.160 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:14.160 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:14.160 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:14.161 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.161 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.161 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:14.161 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:14.161 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.161 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.161 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:14.161 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:14.161 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:14.161 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.161 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.161 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.162 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.162 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.162 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.162 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:14.162 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.162 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:14.162 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:14.162 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.162 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:14.162 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.162 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:14.162 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.163 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.163 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.163 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:14.163 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.163 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:14.163 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.163 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_io_pp 2023-10-06T13:54:14.163 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91649 2023-10-06T13:54:14.163 INFO:tasks.workunit.client.0.smithi019.stdout:test api_io_pp on pid 91649 2023-10-06T13:54:14.167 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_io_pp on pid 91649' 2023-10-06T13:54:14.168 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91649 2023-10-06T13:54:14.168 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2023-10-06T13:54:14.168 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.168 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.168 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_asio 2023-10-06T13:54:14.168 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_asio' 2023-10-06T13:54:14.168 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_asio 2023-10-06T13:54:14.168 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.168 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.168 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.168 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:14.168 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:14.169 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:14.169 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.169 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:14.169 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.169 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:14.169 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:14.169 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.169 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.169 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.169 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:14.169 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.169 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.170 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:14.170 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.170 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:14.170 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.170 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:14.170 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.170 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:14.170 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:14.170 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:14.170 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:14.170 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:14.171 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:14.171 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.171 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:14.171 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:14.171 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.171 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.171 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:14.171 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:14.171 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.171 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:14.171 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:14.171 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:14.172 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:14.172 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.172 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.172 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:14.172 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.172 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.172 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:14.172 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:14.172 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:14.172 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.172 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:14.172 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.173 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:14.173 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.173 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.173 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:14.173 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.173 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.173 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:14.173 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.173 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:14.173 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:14.173 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.173 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:14.173 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.174 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:14.174 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.174 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.174 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.174 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:14.174 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.174 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:14.174 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.174 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_asio 2023-10-06T13:54:14.174 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91663 2023-10-06T13:54:14.175 INFO:tasks.workunit.client.0.smithi019.stdout:test api_asio on pid 91663 2023-10-06T13:54:14.180 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_asio on pid 91663' 2023-10-06T13:54:14.181 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91663 2023-10-06T13:54:14.181 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.181 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.181 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2023-10-06T13:54:14.181 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.181 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:14.181 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.181 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:14.181 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:14.181 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_list 2023-10-06T13:54:14.181 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_list' 2023-10-06T13:54:14.182 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:14.182 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_list 2023-10-06T13:54:14.182 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.182 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.182 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.182 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:14.182 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:14.182 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:14.182 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.182 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:14.182 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.182 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:14.183 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.183 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:14.183 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.183 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.183 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.183 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.183 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:14.183 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.183 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:14.183 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:14.183 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:14.183 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:14.184 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.184 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:14.184 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.184 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:14.184 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.184 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:14.184 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:14.184 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.184 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:14.184 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:14.184 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.184 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.184 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:14.185 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.185 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:14.185 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:14.185 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:14.185 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:14.185 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:14.185 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:14.185 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:14.185 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:14.185 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:14.185 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:14.185 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.186 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.186 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.186 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:14.186 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:14.186 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.186 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:14.186 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.186 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:14.186 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:14.186 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.186 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:14.187 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:14.187 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.187 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:14.187 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.187 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.187 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.187 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.187 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.187 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:14.187 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.187 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:14.187 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.188 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.188 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:14.188 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:14.188 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.188 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:14.188 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:14.188 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:14.188 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:14.188 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.188 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:14.188 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.189 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.189 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:14.189 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:14.189 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.189 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:14.189 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.189 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.189 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.189 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:14.189 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.189 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:14.189 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:14.189 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.190 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91621/exe 2023-10-06T13:54:14.190 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.190 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:14.190 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.190 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_list 2023-10-06T13:54:14.190 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:14.190 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:14.190 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91679 2023-10-06T13:54:14.190 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_list on pid 91679' 2023-10-06T13:54:14.190 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2023-10-06T13:54:14.191 INFO:tasks.workunit.client.0.smithi019.stdout:test api_list on pid 91679 2023-10-06T13:54:14.197 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:14.197 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:14.197 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91679 2023-10-06T13:54:14.197 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.197 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.197 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_lock 2023-10-06T13:54:14.197 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_lock' 2023-10-06T13:54:14.197 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:14.197 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:14.197 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:14.198 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:14.198 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:14.198 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:14.198 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:14.198 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.198 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:14.198 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:14.198 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:14.198 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.198 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.198 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:14.199 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:14.199 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:14.199 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:14.199 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_lock 2023-10-06T13:54:14.199 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.199 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.199 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.199 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:14.199 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.199 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:14.199 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.199 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.200 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:14.200 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:14.200 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:14.200 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:14.200 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.200 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.200 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:14.200 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:14.200 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.200 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.200 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_aio 2023-10-06T13:54:14.201 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:14.201 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:14.201 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.201 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_aio.log 2023-10-06T13:54:14.201 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:14.201 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.201 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:14.201 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.201 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:14.201 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_aio: /' 2023-10-06T13:54:14.201 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:14.201 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.202 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:14.202 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:14.202 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:14.202 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.202 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.202 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:14.202 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:14.202 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.202 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.202 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:14.202 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:14.203 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:14.203 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:14.203 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.203 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.203 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:14.203 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:14.203 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.203 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:14.203 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:14.203 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:14.203 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:14.203 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.204 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:14.204 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.204 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:14.204 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:14.204 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:14.204 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.204 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:14.204 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:14.204 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91628/exe 2023-10-06T13:54:14.204 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.204 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:14.204 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:14.205 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:14.205 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.205 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:14.205 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.205 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:14.205 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.205 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.205 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_lock 2023-10-06T13:54:14.205 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.205 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.205 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.205 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:14.205 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:14.206 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91698 2023-10-06T13:54:14.206 INFO:tasks.workunit.client.0.smithi019.stdout:test api_lock on pid 91698 2023-10-06T13:54:14.212 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_lock on pid 91698' 2023-10-06T13:54:14.212 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91698 2023-10-06T13:54:14.212 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.213 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.213 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.213 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:14.213 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.213 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.213 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:14.213 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:14.213 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:14.213 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2023-10-06T13:54:14.213 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_lock_pp 2023-10-06T13:54:14.213 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:14.213 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.214 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:14.214 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_lock_pp' 2023-10-06T13:54:14.214 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:14.214 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.214 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:14.214 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.214 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.214 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:14.214 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.214 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:14.214 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_lock_pp 2023-10-06T13:54:14.214 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.215 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:14.215 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.215 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:14.215 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:14.215 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.215 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:14.215 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:14.215 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:14.215 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:14.215 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:14.215 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:14.216 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:14.216 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:14.216 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:14.216 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:14.216 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:14.216 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:14.216 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:14.216 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.216 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.216 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.216 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.216 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:14.217 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:14.217 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.217 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.217 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:14.217 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:14.217 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:14.217 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.217 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:14.217 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.217 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.217 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:14.217 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:14.218 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.218 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.218 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:14.218 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:14.218 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.218 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.218 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:14.218 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.218 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.218 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.218 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:14.219 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.219 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:14.219 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.219 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:14.219 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:14.219 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.219 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:14.219 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.219 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:14.219 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:14.219 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.219 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.220 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_aio_pp 2023-10-06T13:54:14.220 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91637/exe 2023-10-06T13:54:14.220 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.220 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:14.220 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_aio_pp.log 2023-10-06T13:54:14.220 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.220 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:14.220 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.220 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:14.220 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:14.220 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:14.220 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:14.221 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_aio_pp: /' 2023-10-06T13:54:14.221 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:14.221 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.221 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.221 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:14.221 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:14.221 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.221 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.221 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:14.221 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:14.221 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:14.221 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.222 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.222 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.222 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.222 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:14.222 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.222 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:14.222 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.222 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:14.222 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_lock_pp 2023-10-06T13:54:14.222 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91718 2023-10-06T13:54:14.222 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_lock_pp on pid 91718' 2023-10-06T13:54:14.223 INFO:tasks.workunit.client.0.smithi019.stdout:test api_lock_pp on pid 91718 2023-10-06T13:54:14.233 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91718 2023-10-06T13:54:14.233 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.233 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.233 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_misc 2023-10-06T13:54:14.233 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.233 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:14.233 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:14.234 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.234 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:14.234 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:14.234 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_misc' 2023-10-06T13:54:14.234 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:14.234 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:14.234 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:14.234 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:14.234 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:14.234 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:14.234 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:14.235 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:14.235 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:14.235 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:14.235 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_misc 2023-10-06T13:54:14.235 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2023-10-06T13:54:14.235 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_io 2023-10-06T13:54:14.235 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_io: /' 2023-10-06T13:54:14.235 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:14.235 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:14.235 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:14.235 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:14.235 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:14.236 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:14.236 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:14.236 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:14.236 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.236 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_io.log 2023-10-06T13:54:14.236 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.236 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:14.236 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:14.236 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:14.236 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.236 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.237 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:14.237 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:14.237 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:14.237 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.237 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:14.237 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:14.237 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:14.237 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.237 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:14.237 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:14.237 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.237 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.238 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:14.238 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:14.238 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.238 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.238 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:14.238 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:14.238 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:14.238 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:14.238 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:14.238 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.238 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:14.238 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:14.239 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.239 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.239 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:14.239 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:14.239 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.239 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:14.239 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:14.239 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:14.239 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.239 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.239 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.239 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:14.240 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:14.240 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.240 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.240 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:14.240 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.240 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.240 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:14.240 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.240 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:14.240 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:14.240 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:14.240 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.241 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.241 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.241 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.241 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.241 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:14.241 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.241 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:14.241 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:14.241 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:14.241 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.241 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.241 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:14.242 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.242 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:14.242 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.242 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:14.242 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:14.242 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.242 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:14.242 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.242 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:14.242 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:14.242 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.243 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.243 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.243 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.243 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:14.243 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.243 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.243 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.243 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:14.243 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.243 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:14.243 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:14.243 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:14.244 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:14.244 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.244 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:14.244 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.244 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:14.244 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.244 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.244 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.244 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:14.244 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.244 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:14.245 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:14.245 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.245 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:14.245 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:14.245 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:14.245 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:14.245 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:14.245 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.245 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:14.245 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91649/exe 2023-10-06T13:54:14.245 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.245 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:14.246 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:14.246 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:14.246 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.246 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.246 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:14.246 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.246 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:14.246 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:14.246 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:14.246 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:14.246 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:14.246 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.247 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.247 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:14.247 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.247 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:14.247 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:14.247 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.247 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.247 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_misc 2023-10-06T13:54:14.247 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:14.247 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:14.247 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.247 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91742 2023-10-06T13:54:14.248 INFO:tasks.workunit.client.0.smithi019.stdout:test api_misc on pid 91742 2023-10-06T13:54:14.255 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_misc on pid 91742' 2023-10-06T13:54:14.255 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91742 2023-10-06T13:54:14.255 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.255 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.255 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:14.255 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.255 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:14.255 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_misc_pp 2023-10-06T13:54:14.255 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:14.255 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.256 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:14.256 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.256 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:14.256 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:14.256 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_misc_pp' 2023-10-06T13:54:14.256 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2023-10-06T13:54:14.256 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:14.256 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:14.256 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:14.256 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:14.256 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:14.256 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:14.257 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:14.257 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:14.257 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:14.257 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:14.257 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_io_pp 2023-10-06T13:54:14.257 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.257 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.257 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_misc_pp 2023-10-06T13:54:14.257 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:14.257 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.257 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.257 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:14.258 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.258 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.258 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:14.258 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.258 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:14.258 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.258 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:14.258 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_io_pp.log 2023-10-06T13:54:14.258 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:14.258 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:14.258 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91663/exe 2023-10-06T13:54:14.259 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.259 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:14.259 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.259 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:14.259 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_io_pp: /' 2023-10-06T13:54:14.259 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:14.259 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:14.259 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.259 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:14.259 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.259 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:14.259 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.260 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:14.260 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:14.260 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:14.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:14.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:14.260 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:14.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:14.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.261 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:14.261 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:14.261 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.261 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:14.261 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.261 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:14.261 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:14.261 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:14.261 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:14.261 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:14.261 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:14.262 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.262 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:14.262 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.262 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:14.262 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:14.262 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.262 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:14.262 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.262 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:14.262 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:14.262 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.262 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:14.263 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.263 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:14.263 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:14.263 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:14.263 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:14.263 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:14.263 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:14.263 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:14.263 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:14.263 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:14.263 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:14.263 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:14.264 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:14.264 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_misc_pp 2023-10-06T13:54:14.264 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91762 2023-10-06T13:54:14.264 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_misc_pp on pid 91762' 2023-10-06T13:54:14.264 INFO:tasks.workunit.client.0.smithi019.stdout:test api_misc_pp on pid 91762 2023-10-06T13:54:14.271 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91762 2023-10-06T13:54:14.271 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.271 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_asio.log 2023-10-06T13:54:14.271 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.272 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_tier_pp 2023-10-06T13:54:14.272 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_tier_pp' 2023-10-06T13:54:14.272 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.272 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:14.272 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2023-10-06T13:54:14.272 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.272 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:14.272 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_tier_pp 2023-10-06T13:54:14.272 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:14.272 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_asio: /' 2023-10-06T13:54:14.272 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.272 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:14.273 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.273 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:14.273 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:14.273 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:14.273 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.273 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.273 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:14.273 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:14.273 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:14.273 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.273 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:14.274 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.274 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:14.274 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:14.274 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.274 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.274 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.274 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.274 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.274 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.274 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.274 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.274 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.275 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.275 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:14.275 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.275 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:14.275 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:14.275 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.275 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:14.275 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.275 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:14.275 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.275 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.275 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.276 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.276 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:14.276 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:14.276 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.276 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:14.276 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.276 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.276 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.276 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:14.276 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.276 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:14.276 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:14.277 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.277 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_asio 2023-10-06T13:54:14.277 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.277 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:14.277 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.277 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:14.277 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:14.277 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.277 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:14.277 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.277 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:14.278 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.278 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:14.278 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.278 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:14.278 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:14.278 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.278 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:14.278 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.278 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:14.278 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:14.278 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:14.278 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:14.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:14.279 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_tier_pp 2023-10-06T13:54:14.279 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.279 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:14.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:14.279 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.279 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91777 2023-10-06T13:54:14.280 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_tier_pp on pid 91777' 2023-10-06T13:54:14.280 INFO:tasks.workunit.client.0.smithi019.stdout:test api_tier_pp on pid 91777 2023-10-06T13:54:14.293 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91777 2023-10-06T13:54:14.293 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.293 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.293 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_pool 2023-10-06T13:54:14.293 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:14.294 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:14.294 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:14.294 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_pool' 2023-10-06T13:54:14.294 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:14.294 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.294 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:14.294 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.294 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:14.294 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:14.294 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:14.294 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.294 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:14.295 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.295 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:14.295 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:14.295 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2023-10-06T13:54:14.295 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:14.295 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.295 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:14.295 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.295 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:14.295 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.295 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:14.296 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:14.296 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:14.296 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.296 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:14.296 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.296 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:14.296 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:14.296 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.296 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.296 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.296 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.296 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:14.296 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.297 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.297 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.297 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.297 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.297 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:14.297 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.297 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.297 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:14.297 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:14.297 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91679/exe 2023-10-06T13:54:14.297 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.297 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:14.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:14.298 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:14.298 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.298 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:14.298 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:14.298 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:14.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:14.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:14.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:14.299 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:14.299 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:14.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:14.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:14.299 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:14.299 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:14.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.300 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:14.300 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.300 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:14.300 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.300 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.300 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:14.300 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:14.300 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.300 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:14.300 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.300 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:14.300 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.301 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.301 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:14.301 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.301 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:14.301 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.301 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:14.301 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.301 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:14.301 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:14.301 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:14.301 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:14.301 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:14.302 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:14.302 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:14.302 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:14.302 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:14.302 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:14.302 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_list.log 2023-10-06T13:54:14.302 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91698/exe 2023-10-06T13:54:14.302 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_list 2023-10-06T13:54:14.302 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:14.302 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:14.302 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:14.303 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.303 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:14.303 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.303 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:14.303 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:14.303 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:14.303 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.303 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:14.303 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.303 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:14.303 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:14.303 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:14.304 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.304 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:14.304 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.304 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:14.304 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.304 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:14.304 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:14.304 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:14.304 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:14.304 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.304 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:14.304 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:14.305 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.305 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:14.305 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:14.305 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:14.305 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:14.305 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.305 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.305 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:14.305 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.305 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:14.305 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:14.305 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.306 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:14.306 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:14.306 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:14.306 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:14.306 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:14.306 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:14.306 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:14.306 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:14.306 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:14.306 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:14.306 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_lock.log 2023-10-06T13:54:14.306 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_lock 2023-10-06T13:54:14.307 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:14.307 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:14.307 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:14.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:14.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:14.307 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:14.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:14.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:14.308 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.308 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:14.308 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.308 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:14.308 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:14.308 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:14.308 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_pool 2023-10-06T13:54:14.308 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_list: /' 2023-10-06T13:54:14.308 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_pool 2023-10-06T13:54:14.308 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.308 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:14.308 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.309 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:14.309 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:14.309 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91808 2023-10-06T13:54:14.309 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_pool on pid 91808' 2023-10-06T13:54:14.309 INFO:tasks.workunit.client.0.smithi019.stdout:test api_pool on pid 91808 2023-10-06T13:54:14.327 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91808 2023-10-06T13:54:14.327 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.327 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.327 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:14.327 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:14.327 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2023-10-06T13:54:14.327 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:14.327 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.327 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:14.327 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.328 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:14.328 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:14.328 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.328 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:14.328 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.328 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:14.328 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.328 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:14.328 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.328 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:14.328 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:14.328 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.329 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.329 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.329 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.329 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.329 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.329 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:14.329 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.329 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:14.329 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.329 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:14.329 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.329 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:14.330 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:14.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.330 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:14.330 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:14.330 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:14.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:14.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:14.330 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:14.331 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.331 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_lock: /' 2023-10-06T13:54:14.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:14.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:14.331 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:14.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:14.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:14.331 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.332 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.332 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.332 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:14.332 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.332 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:14.332 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:14.332 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:14.332 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:14.332 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.332 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:14.332 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.332 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:14.332 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:14.333 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.333 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:14.333 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.333 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:14.333 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.333 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.333 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:14.333 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.333 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:14.333 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:14.333 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_snapshots 2023-10-06T13:54:14.334 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_snapshots' 2023-10-06T13:54:14.334 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:14.334 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:14.334 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:14.334 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.334 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:14.334 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.334 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:14.334 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:14.334 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.334 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.334 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:14.335 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.335 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:14.335 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.335 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:14.335 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:14.335 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.335 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:14.335 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.335 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:14.335 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:14.335 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:14.335 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:14.336 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:14.336 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91762/exe 2023-10-06T13:54:14.336 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:14.336 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.336 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:14.336 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.336 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:14.336 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:14.336 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:14.336 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:14.336 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.336 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:14.337 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.337 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:14.337 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:14.337 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:14.337 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.337 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:14.337 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.337 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:14.337 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.337 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.337 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:14.337 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.338 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:14.338 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:14.338 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:14.338 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:14.338 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:14.338 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:14.338 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.338 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:14.338 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.338 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:14.338 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:14.338 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:14.339 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.339 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:14.339 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.339 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:14.339 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.339 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:14.339 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.339 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:14.339 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:14.339 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:14.339 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:14.339 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:14.340 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.340 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:14.340 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.340 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:14.340 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:14.340 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.340 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.340 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:14.340 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.340 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:14.340 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.340 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:14.341 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:14.341 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.341 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:14.341 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:14.341 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:14.341 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.341 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:14.341 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:14.341 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:14.341 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:14.341 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:14.341 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:14.342 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:14.342 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:14.342 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:14.342 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:14.342 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_snapshots 2023-10-06T13:54:14.342 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:14.342 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:14.342 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:14.342 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.342 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:14.342 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.342 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:14.343 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:14.343 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.343 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_misc_pp: /' 2023-10-06T13:54:14.343 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.343 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:14.343 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.343 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:14.343 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.343 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:14.343 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:14.343 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.343 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:14.344 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.344 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:14.344 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:14.344 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_misc_pp.log 2023-10-06T13:54:14.344 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91718/exe 2023-10-06T13:54:14.344 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:14.344 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91742/exe 2023-10-06T13:54:14.344 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.344 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:14.344 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:14.344 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:14.344 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:14.345 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:14.345 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.345 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.345 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:14.345 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:14.345 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.345 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:14.345 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:14.345 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:14.345 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:14.345 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:14.345 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.346 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:14.346 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:14.346 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:14.346 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:14.346 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:14.346 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:14.346 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:14.346 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:14.346 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:14.346 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:14.346 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:14.346 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:14.347 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:14.347 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.347 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:14.347 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:14.347 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:14.347 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:14.347 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:14.347 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:14.347 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_lock_pp.log 2023-10-06T13:54:14.347 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:14.347 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_snapshots 2023-10-06T13:54:14.347 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_lock_pp 2023-10-06T13:54:14.348 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91883 2023-10-06T13:54:14.348 INFO:tasks.workunit.client.0.smithi019.stdout:test api_snapshots on pid 91883 2023-10-06T13:54:14.351 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_snapshots on pid 91883' 2023-10-06T13:54:14.351 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_misc.log 2023-10-06T13:54:14.351 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91883 2023-10-06T13:54:14.351 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.351 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.351 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_snapshots_pp 2023-10-06T13:54:14.351 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_snapshots_pp' 2023-10-06T13:54:14.351 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:14.352 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:14.352 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:14.352 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.352 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:14.352 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.352 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:14.352 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:14.352 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.352 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.352 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:14.352 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.352 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:14.353 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.353 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:14.353 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_misc_pp 2023-10-06T13:54:14.353 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:14.353 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.353 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:14.353 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.353 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:14.353 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:14.353 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.353 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_snapshots_pp 2023-10-06T13:54:14.353 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_misc: /' 2023-10-06T13:54:14.353 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_snapshots_pp 2023-10-06T13:54:14.354 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91893 2023-10-06T13:54:14.354 INFO:tasks.workunit.client.0.smithi019.stdout:test api_snapshots_pp on pid 91893 2023-10-06T13:54:14.363 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_snapshots_pp on pid 91893' 2023-10-06T13:54:14.363 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91893 2023-10-06T13:54:14.363 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.363 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.363 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91777/exe 2023-10-06T13:54:14.363 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:14.363 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:14.363 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:14.364 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:14.364 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.364 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:14.364 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.364 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:14.364 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:14.364 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.364 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:14.364 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_lock_pp: /' 2023-10-06T13:54:14.364 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.364 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:14.364 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.365 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:14.365 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:14.365 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.365 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:14.365 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:14.365 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:14.365 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:14.365 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:14.365 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:14.365 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:14.365 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:14.365 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:14.366 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:14.366 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:14.366 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:14.366 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_misc 2023-10-06T13:54:14.366 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_tier_pp 2023-10-06T13:54:14.366 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_tier_pp: /' 2023-10-06T13:54:14.366 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2023-10-06T13:54:14.366 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_tier_pp.log 2023-10-06T13:54:14.366 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.366 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:14.366 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.367 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:14.367 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:14.367 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.367 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.367 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.367 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.367 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.367 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.367 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:14.367 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:14.367 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:14.367 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:14.368 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:14.368 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:14.368 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.368 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:14.368 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:14.368 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:14.368 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:14.368 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:14.368 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.368 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:14.368 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.368 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:14.368 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:14.369 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.369 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:14.369 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.369 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:14.369 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:14.369 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.369 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.369 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.369 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.369 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:14.369 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.369 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:14.370 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:14.370 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.370 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:14.370 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.370 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:14.370 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.370 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:14.370 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.370 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:14.370 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.742 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:14.742 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:14.742 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91808/exe 2023-10-06T13:54:14.742 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:14.742 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2023-10-06T13:54:14.742 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_stat 2023-10-06T13:54:14.742 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_stat' 2023-10-06T13:54:14.742 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:14.743 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:14.743 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:14.743 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:14.743 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:14.743 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:14.743 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:14.743 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:14.743 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:14.743 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:14.744 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.744 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.744 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.744 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.744 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.744 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.744 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_stat 2023-10-06T13:54:14.745 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:14.745 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_pool 2023-10-06T13:54:14.745 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_stat 2023-10-06T13:54:14.745 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91955 2023-10-06T13:54:14.745 INFO:tasks.workunit.client.0.smithi019.stdout:test api_stat on pid 91955 2023-10-06T13:54:14.746 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_stat on pid 91955' 2023-10-06T13:54:14.746 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91955 2023-10-06T13:54:14.746 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:14.746 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:14.746 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.747 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2023-10-06T13:54:14.747 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [----------] Global test environment set-up. 2023-10-06T13:54:14.747 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [----------] 12 tests from AsioRados 2023-10-06T13:54:14.747 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.245+0000 7f55c9d3c780 10 monclient: get_monmap_and_config 2023-10-06T13:54:14.747 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.245+0000 7f55c9d3c780 10 monclient: build_initial_monmap 2023-10-06T13:54:14.747 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.245+0000 7f55c9d3c780 10 monclient: monmap: 2023-10-06T13:54:14.747 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: epoch 0 2023-10-06T13:54:14.747 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:54:14.747 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: last_changed 0.000000 2023-10-06T13:54:14.747 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: created 0.000000 2023-10-06T13:54:14.747 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: min_mon_release 0 (unknown) 2023-10-06T13:54:14.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: election_strategy: 1 2023-10-06T13:54:14.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:54:14.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:54:14.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:54:14.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.246+0000 7f55c9d3c780 1 Processor -- start 2023-10-06T13:54:14.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.248+0000 7f55c9d3c780 1 -- start start 2023-10-06T13:54:14.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.248+0000 7f55c9d3c780 10 monclient: init 2023-10-06T13:54:14.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 10 monclient: _reopen_session rank -1 2023-10-06T13:54:14.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T13:54:14.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5929d30c0 0x55d592a16310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:54:14.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 10 monclient(hunting): picked mon.noname-b con 0x55d5929d30c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:54:14.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x55d592aa2e10 0x55d592aaea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:54:14.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 10 monclient(hunting): picked mon.noname-c con 0x55d592aa2e10 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:54:14.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d592aa5bc0 0x55d592a16e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:54:14.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 10 monclient(hunting): picked mon.noname-a con 0x55d592aa5bc0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:54:14.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 10 monclient(hunting): start opening mon connection 2023-10-06T13:54:14.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d592928280 con 0x55d592aa5bc0 2023-10-06T13:54:14.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 10 monclient(hunting): start opening mon connection 2023-10-06T13:54:14.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d592927f20 con 0x55d5929d30c0 2023-10-06T13:54:14.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 10 monclient(hunting): start opening mon connection 2023-10-06T13:54:14.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d59290b420 con 0x55d592aa2e10 2023-10-06T13:54:14.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 10 monclient(hunting): _renew_subs 2023-10-06T13:54:14.750 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.249+0000 7f55c9d3c780 10 monclient(hunting): authenticate will time out at 1382.435782s 2023-10-06T13:54:14.761 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_pool: /' 2023-10-06T13:54:14.761 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2023-10-06T13:54:14.761 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_pool.log 2023-10-06T13:54:14.761 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:14.761 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:14.761 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:14.761 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:14.761 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.761 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:14.761 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:14.762 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.762 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.762 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.762 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:14.762 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.762 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:14.762 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:14.762 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:14.762 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:14.762 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:14.762 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:14.763 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:14.763 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:14.763 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.049 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:15.049 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.050 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:15.050 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.050 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:15.050 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.050 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.050 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.050 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.051 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.051 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.051 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.051 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:15.051 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.052 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:15.052 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.052 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.052 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.052 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.052 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.052 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.052 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.052 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.052 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.052 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.052 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.053 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.053 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.053 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.053 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.053 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.053 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.053 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.053 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.054 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.054 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.054 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.054 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.054 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.054 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.054 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.054 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.054 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_stat_pp 2023-10-06T13:54:15.054 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_stat_pp' 2023-10-06T13:54:15.054 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.055 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.055 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.055 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.055 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.055 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.055 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.055 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.055 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.055 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.055 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.055 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.056 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.056 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.056 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.056 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.056 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.056 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.056 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.056 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.056 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.056 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.056 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.056 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.057 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.057 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.057 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.057 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.057 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.057 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.057 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.057 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.057 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.057 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.057 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.058 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.058 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.058 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.058 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.058 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.058 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_stat_pp 2023-10-06T13:54:15.058 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.058 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.058 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.058 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.058 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.058 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.059 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.059 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.059 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.059 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.059 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.059 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.059 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.059 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.059 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.059 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.059 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:15.059 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.060 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.060 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.060 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.060 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.060 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.060 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.060 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.060 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.060 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.060 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.060 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.060 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.061 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.061 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.061 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.061 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.061 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.061 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.061 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.061 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.061 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_stat_pp 2023-10-06T13:54:15.061 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.061 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91999 2023-10-06T13:54:15.061 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.061 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.062 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.062 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_stat_pp on pid 91999' 2023-10-06T13:54:15.062 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-test api_stat_pp on pid 91999 2023-10-06T13:54:15.069 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=91999 2023-10-06T13:54:15.069 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:15.070 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:15.070 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.070 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2023-10-06T13:54:15.070 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.070 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.070 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.095 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:15.095 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.096 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.096 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.096 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.096 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.096 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.096 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:15.096 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.096 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:15.096 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:15.096 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.096 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:15.097 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.097 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.097 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.097 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.097 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:15.097 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.097 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.097 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:15.097 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.097 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.097 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.097 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.098 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:15.098 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.098 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.098 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:15.098 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.098 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:15.098 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:15.098 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.098 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:15.098 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.098 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:15.098 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.099 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.099 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.099 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.099 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.099 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.099 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.099 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91883/exe 2023-10-06T13:54:15.099 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.099 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.099 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.099 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.099 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.100 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.100 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.100 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.100 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.100 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.100 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.100 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_snapshots.log 2023-10-06T13:54:15.100 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.100 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.100 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.100 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_snapshots 2023-10-06T13:54:15.100 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.101 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.101 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.101 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.101 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.101 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.101 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_snapshots: /' 2023-10-06T13:54:15.101 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_watch_notify 2023-10-06T13:54:15.101 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_watch_notify' 2023-10-06T13:54:15.101 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:15.101 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_watch_notify 2023-10-06T13:54:15.101 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.101 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.102 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.102 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.102 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.102 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.102 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.102 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.102 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.102 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.102 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.102 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.102 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.102 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.103 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.103 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.103 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.103 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_watch_notify 2023-10-06T13:54:15.103 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92044 2023-10-06T13:54:15.103 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_watch_notify on pid 92044' 2023-10-06T13:54:15.103 INFO:tasks.workunit.client.0.smithi019.stdout:test api_watch_notify on pid 92044 2023-10-06T13:54:15.108 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=92044 2023-10-06T13:54:15.205 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:15.205 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:15.205 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_watch_notify_pp 2023-10-06T13:54:15.205 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_watch_notify_pp' 2023-10-06T13:54:15.205 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.205 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2023-10-06T13:54:15.205 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:15.206 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_watch_notify_pp 2023-10-06T13:54:15.206 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.206 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.206 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.206 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.206 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.206 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.206 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.206 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.206 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.206 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.207 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.207 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.207 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.207 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.207 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.207 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.207 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.207 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.207 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.207 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.207 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.208 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.208 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.208 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.208 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.208 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.208 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.208 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.208 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.208 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.208 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.208 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.208 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.209 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.209 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.209 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.209 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:15.209 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:15.209 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.209 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.209 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.209 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.209 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.210 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:15.210 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:15.210 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.210 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:15.210 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.210 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.210 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:15.210 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_watch_notify_pp 2023-10-06T13:54:15.210 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:15.211 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92056 2023-10-06T13:54:15.211 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_watch_notify_pp on pid 92056' 2023-10-06T13:54:15.211 INFO:tasks.workunit.client.0.smithi019.stdout:test api_watch_notify_pp on pid 92056 2023-10-06T13:54:15.222 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=92056 2023-10-06T13:54:15.222 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:15.222 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:15.222 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2023-10-06T13:54:15.222 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:15.222 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.222 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.222 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:15.222 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.222 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:15.223 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.223 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.223 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.223 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.223 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.223 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:15.223 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.223 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:15.223 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.223 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:15.223 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.223 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.224 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.224 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.224 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.224 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.224 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.224 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.224 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.224 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.224 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:15.224 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.224 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.224 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.225 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.225 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.225 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.225 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.225 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:15.225 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.225 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:15.225 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.225 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.225 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.225 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.225 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.226 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:15.226 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.226 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.226 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.226 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.226 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.226 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.226 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.226 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:15.226 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_cmd 2023-10-06T13:54:15.227 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_cmd' 2023-10-06T13:54:15.227 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.227 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:15.227 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.227 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.227 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.227 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.227 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:15.227 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.227 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:15.228 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.228 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.228 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.228 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.228 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.228 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.228 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.228 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.229 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.229 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.229 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:15.229 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.229 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:15.229 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91955/exe 2023-10-06T13:54:15.229 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.229 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:15.229 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.229 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.230 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.230 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.230 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:15.230 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.230 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:15.230 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.230 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.230 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.231 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.231 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.231 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.231 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.231 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.231 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.231 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.231 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.231 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.231 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.232 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.232 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.232 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_cmd 2023-10-06T13:54:15.232 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.232 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.232 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.232 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.232 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.232 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.232 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.232 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.233 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.233 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.233 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.233 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.233 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.233 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.233 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.233 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.233 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.233 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.233 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.233 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:15.234 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.234 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.234 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.234 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.234 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.234 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.234 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.234 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.235 INFO:tasks.workunit.client.0.smithi019.stdout:06T13:54:14.254+0000 7f55c783b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5929d30c0 0x55d592a16310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:54:15.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.254+0000 7f55c783b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5929d30c0 0x55d592a16310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:59694/0 (socket says 172.21.15.19:59694) 2023-10-06T13:54:15.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.254+0000 7f55c783b640 1 -- 172.21.15.19:0/3678909562 learned_addr learned my addr 172.21.15.19:0/3678909562 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:54:15.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.254+0000 7f55c783b640 10 monclient(hunting): get_auth_request con 0x55d5929d30c0 auth_method 0 2023-10-06T13:54:15.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.254+0000 7f55c783b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:54:15.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.254+0000 7f55c783b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:54:15.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.254+0000 7f55c783b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:54:15.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.255+0000 7f55c783b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:54:15.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.255+0000 7f55c783b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:54:15.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.255+0000 7f55c783b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:54:15.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.258+0000 7f55c783b640 10 monclient(hunting): handle_auth_done global_id 4432 payload 274 2023-10-06T13:54:15.236 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.258+0000 7f55c783b640 1 -- 172.21.15.19:0/3678909562 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x55d592aa2e10 msgr2=0x55d592aaea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:54:15.236 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.258+0000 7f55c783b640 1 --2- 172.21.15.19:0/3678909562 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x55d592aa2e10 0x55d592aaea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.236 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.258+0000 7f55c783b640 1 -- 172.21.15.19:0/3678909562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d592aa5bc0 msgr2=0x55d592a16e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:54:15.236 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.258+0000 7f55c783b640 1 --2- 172.21.15.19:0/3678909562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d592aa5bc0 0x55d592a16e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.236 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.258+0000 7f55c783b640 10 monclient: _finish_hunting 0 2023-10-06T13:54:15.236 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.258+0000 7f55c783b640 1 monclient: found mon.noname-b 2023-10-06T13:54:15.236 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.258+0000 7f55c783b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:54:15.236 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.258+0000 7f55c783b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:54:15.236 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.258+0000 7f55c783b640 1 -- 172.21.15.19:0/3678909562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc005d20 con 0x55d5929d30c0 2023-10-06T13:54:15.236 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.272+0000 7f55c783b640 1 --2- 172.21.15.19:0/3678909562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5929d30c0 0x55d592a16310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f55b800a5e0 tx=0x7f55b8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ede0188e6f5dd81 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:54:15.237 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.237 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.237 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.237 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.238 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.238 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.238 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.238 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c6038640 1 -- 172.21.15.19:0/3678909562 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8008a40 con 0x55d5929d30c0 2023-10-06T13:54:15.238 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c6038640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:54:15.238 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c6038640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:54:15.238 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c6038640 10 monclient: dump: 2023-10-06T13:54:15.238 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: epoch 1 2023-10-06T13:54:15.238 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:54:15.238 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.239 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.239 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: min_mon_release 18 (reef) 2023-10-06T13:54:15.239 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: election_strategy: 1 2023-10-06T13:54:15.239 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:54:15.239 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:54:15.239 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:54:15.239 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.239 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c6038640 10 monclient: _finish_auth 0 2023-10-06T13:54:15.239 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c6038640 10 monclient: _check_auth_tickets 2023-10-06T13:54:15.239 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c6038640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:15.239 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c6038640 1 -- 172.21.15.19:0/3678909562 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b8008bc0 con 0x55d5929d30c0 2023-10-06T13:54:15.240 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c6038640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:54:15.240 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c6038640 1 -- 172.21.15.19:0/3678909562 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8013440 con 0x55d5929d30c0 2023-10-06T13:54:15.240 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c6038640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:54:15.240 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c6038640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:54:15.240 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c6038640 10 monclient: dump: 2023-10-06T13:54:15.240 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: epoch 1 2023-10-06T13:54:15.240 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:54:15.240 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.240 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.240 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: min_mon_release 18 (reef) 2023-10-06T13:54:15.240 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: election_strategy: 1 2023-10-06T13:54:15.240 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:54:15.241 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:54:15.241 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:54:15.241 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.241 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c9d3c780 5 monclient: authenticate success, global_id 4432 2023-10-06T13:54:15.241 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c9d3c780 10 monclient: get_monmap_and_config success 2023-10-06T13:54:15.241 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.311+0000 7f55c9d3c780 4 set_mon_vals no callback set 2023-10-06T13:54:15.241 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.312+0000 7f55c9d3c780 10 monclient: shutdown 2023-10-06T13:54:15.241 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.312+0000 7f55c9d3c780 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:54:15.241 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.312+0000 7f55c9d3c780 1 -- 172.21.15.19:0/3678909562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5929d30c0 msgr2=0x55d592a16310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:54:15.241 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.312+0000 7f55c9d3c780 1 --2- 172.21.15.19:0/3678909562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5929d30c0 0x55d592a16310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f55b800a5e0 tx=0x7f55b8005f90 comp rx=0 tx=0).stop 2023-10-06T13:54:15.243 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_stat 2023-10-06T13:54:15.243 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.256 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.256 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.256 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.256 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.256 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.256 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.256 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_stat.log 2023-10-06T13:54:15.256 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.256 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_cmd 2023-10-06T13:54:15.256 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.257 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.257 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.257 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.257 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.257 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.257 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.257 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.257 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.257 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92082 2023-10-06T13:54:15.258 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.test api_cmd on pid 92082 2023-10-06T13:54:15.259 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.259 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_cmd on pid 92082' 2023-10-06T13:54:15.259 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.259 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=92082 2023-10-06T13:54:15.259 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:15.259 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:15.259 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.259 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.260 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_stat: /' 2023-10-06T13:54:15.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.260 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.260 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.260 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.261 INFO:tasks.workunit.client.0.smithi019.stdout:313+0000 7f55c9d3c780 1 -- 172.21.15.19:0/3678909562 shutdown_connections 2023-10-06T13:54:15.261 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.313+0000 7f55c9d3c780 1 --2- 172.21.15.19:0/3678909562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d592aa5bc0 0x55d592a16e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.261 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.313+0000 7f55c9d3c780 1 --2- 172.21.15.19:0/3678909562 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x55d592aa2e10 0x55d592aaea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.261 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.313+0000 7f55c9d3c780 1 --2- 172.21.15.19:0/3678909562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5929d30c0 0x55d592a16310 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.261 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.313+0000 7f55c9d3c780 1 -- 172.21.15.19:0/3678909562 >> 172.21.15.19:0/3678909562 conn(0x55d592a182a0 msgr2=0x55d592a137e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:54:15.261 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.313+0000 7f55c9d3c780 1 -- 172.21.15.19:0/3678909562 shutdown_connections 2023-10-06T13:54:15.261 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.313+0000 7f55c9d3c780 1 -- 172.21.15.19:0/3678909562 wait complete. 2023-10-06T13:54:15.261 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.314+0000 7f55c9d3c780 10 monclient: build_initial_monmap 2023-10-06T13:54:15.261 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.314+0000 7f55c9d3c780 10 monclient: monmap: 2023-10-06T13:54:15.261 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: epoch 0 2023-10-06T13:54:15.261 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:54:15.261 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: last_changed 0.000000 2023-10-06T13:54:15.262 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: created 0.000000 2023-10-06T13:54:15.262 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: min_mon_release 0 (unknown) 2023-10-06T13:54:15.262 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: election_strategy: 1 2023-10-06T13:54:15.262 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:54:15.262 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:54:15.262 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:54:15.262 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.262 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.314+0000 7f55c9d3c780 1 librados: starting msgr at 2023-10-06T13:54:15.262 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.314+0000 7f55c9d3c780 1 librados: starting objecter 2023-10-06T13:54:15.262 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.314+0000 7f55c9d3c780 1 Processor -- start 2023-10-06T13:54:15.262 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.314+0000 7f55c9d3c780 1 -- start start 2023-10-06T13:54:15.263 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.314+0000 7f55c9d3c780 1 librados: setting wanted keys 2023-10-06T13:54:15.263 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.314+0000 7f55c9d3c780 1 librados: calling monclient init 2023-10-06T13:54:15.263 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.314+0000 7f55c9d3c780 10 monclient: init 2023-10-06T13:54:15.263 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 10 monclient: _reopen_session rank -1 2023-10-06T13:54:15.263 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T13:54:15.263 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5929d30c0 0x55d592aa0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:54:15.263 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 10 monclient(hunting): picked mon.noname-b con 0x55d5929d30c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:54:15.263 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d592aa2e10 0x55d592aa1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:54:15.263 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 10 monclient(hunting): picked mon.noname-a con 0x55d592aa2e10 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:54:15.266 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.266 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.266 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_cmd_pp 2023-10-06T13:54:15.266 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.267 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_cmd_pp' 2023-10-06T13:54:15.267 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.267 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.267 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.267 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.267 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91893/exe 2023-10-06T13:54:15.267 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.267 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.267 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.267 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.267 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.267 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.268 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_cmd_pp 2023-10-06T13:54:15.268 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.268 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.268 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.268 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.268 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.268 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.268 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2023-10-06T13:54:15.268 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.268 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.268 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.268 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.268 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.269 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.269 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.269 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.269 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.269 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.269 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.269 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:15.269 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.269 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.269 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.269 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.270 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x55d592aa5bc0 0x55d592b43600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto api_misc: [==========] Running 11 tests from 4 test suites. 2023-10-06T13:54:15.270 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [----------] Global test environment set-up. 2023-10-06T13:54:15.270 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2023-10-06T13:54:15.270 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2023-10-06T13:54:15.270 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2023-10-06T13:54:15.270 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2023-10-06T13:54:15.270 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:15.270 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [----------] 1 test from LibRadosMiscConnectFailure 2023-10-06T13:54:15.270 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2023-10-06T13:54:15.270 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 0 2023-10-06T13:54:15.271 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.317+0000 7fa9bec0d780 10 monclient: get_monmap_and_config 2023-10-06T13:54:15.271 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.317+0000 7fa9bec0d780 10 monclient: build_initial_monmap 2023-10-06T13:54:15.271 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.317+0000 7fa9bec0d780 10 monclient: monmap: 2023-10-06T13:54:15.271 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: epoch 0 2023-10-06T13:54:15.271 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:54:15.271 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: last_changed 2023-10-06T13:54:14.318584+0000 2023-10-06T13:54:15.271 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: created 2023-10-06T13:54:14.318584+0000 2023-10-06T13:54:15.271 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: min_mon_release 0 (unknown) 2023-10-06T13:54:15.271 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: election_strategy: 1 2023-10-06T13:54:15.271 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T13:54:15.271 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T13:54:15.271 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T13:54:15.272 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T13:54:15.272 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T13:54:15.272 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:15.272 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.319+0000 7fa9bec0d780 1 Processor -- start 2023-10-06T13:54:15.272 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.319+0000 7fa9bec0d780 1 -- start start 2023-10-06T13:54:15.272 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.319+0000 7fa9bec0d780 10 monclient: init 2023-10-06T13:54:15.272 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.320+0000 7fa9bec0d780 10 monclient: _reopen_session rank -1 2023-10-06T13:54:15.272 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.320+0000 7fa9bec0d780 10 monclient: _add_conns ranks=[2,0,4,3,1] 2023-10-06T13:54:15.272 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.320+0000 7fa9bec0d780 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x563be6931c50 0x563be692fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:54:15.272 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.320+0000 7fa9bec0d780 10 monclient(hunting): picked mon.noname-b con 0x563be6931c50 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:54:15.272 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.320+0000 7fa9bec0d780 10 monclient(hunting): picked mon.noname-a-legacy con 0x563be69302a0 addr v1:172.21.15.19:6789/0 2023-10-06T13:54:15.272 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.320+0000 7fa9bec0d780 1 --2- >> v2:172.21.15.83:3300/0 conn(0x563be692a9a0 0x563be6929a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:54:15.273 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.320+0000 7fa9bec0d780 10 monclient(hunting): picked mon.noname-c con 0x563be692a9a0 addr v2:172.21.15.83:3300/0 2023-10-06T13:54:15.273 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.320+0000 7fa9bec0d780 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x563be5cda3f0 con 0x563be69302a0 2023-10-06T13:54:15.273 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.320+0000 7fa9bec0d780 10 monclient(hunting): start opening mon connection 2023-10-06T13:54:15.273 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.321+0000 7fa9bec0d780 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x563be5d3cd40 con 0x563be692a9a0 2023-10-06T13:54:15.273 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.321+0000 7fa9bec0d780 10 monclient(hunting): start opening mon connection 2023-10-06T13:54:15.273 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.321+0000 7fa9bec0d780 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x563be5cd8280 con 0x563be6931c50 2023-10-06T13:54:15.273 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.321+0000 7fa9bec0d780 10 monclient(hunting): _renew_subs 2023-10-06T13:54:15.277 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.277 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.277 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.277 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.277 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.277 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.277 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.277 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.278 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.278 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.278 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.278 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.278 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:15.278 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.278 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.278 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.278 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.278 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.278 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:15.278 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:15.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:15.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:15.279 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.279 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.279 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.279 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.280 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:15.280 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.280 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.280 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.280 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:15.280 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.280 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:15.280 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.280 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:15.280 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.280 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:15.280 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_cmd_pp 2023-10-06T13:54:15.281 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:15.281 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.281 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:15.281 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.281 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:15.281 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92111 2023-10-06T13:54:15.281 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54test api_cmd_pp on pid 92111 2023-10-06T13:54:15.296 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_cmd_pp on pid 92111' 2023-10-06T13:54:15.296 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=92111 2023-10-06T13:54:15.296 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.296 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.296 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:15.297 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:15.297 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.297 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.297 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.297 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.297 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.297 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2023-10-06T13:54:15.297 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.297 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2023-10-06T13:54:15.297 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.297 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_snapshots_pp: /' 2023-10-06T13:54:15.298 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.298 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/91999/exe 2023-10-06T13:54:15.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.298 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.298 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.299 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_service 2023-10-06T13:54:15.299 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.299 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_service' 2023-10-06T13:54:15.299 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:15.299 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:15.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:15.299 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.299 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:15.300 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.300 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:15.300 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:15.300 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.300 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.300 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.300 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.300 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.300 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:15.300 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:15.300 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_snapshots_pp 2023-10-06T13:54:15.300 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.301 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:15.301 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.301 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:15.301 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:15.301 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.301 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.301 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:15.301 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.301 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:15.301 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.301 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.301 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.302 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.302 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.302 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.302 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.302 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.302 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.302 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.302 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.302 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.302 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.302 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.302 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.303 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.303 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.303 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.303 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.303 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.303 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.303 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.303 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.303 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.303 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.303 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.304 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.304 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.304 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.304 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.304 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.304 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.304 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.304 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.304 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_stat_pp: /' 2023-10-06T13:54:15.304 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.304 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.304 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.305 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_service 2023-10-06T13:54:15.305 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.305 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.305 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.305 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.305 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.305 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.305 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.305 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.305 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.305 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.305 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.306 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_stat_pp.log 2023-10-06T13:54:15.306 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.306 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.306 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.306 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.306 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.306 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.306 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.306 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/92056/exe 2023-10-06T13:54:15.306 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.306 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.306 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.307 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.307 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.307 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.307 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.307 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.308 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.308 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.308 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.308 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.308 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.308 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.308 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.308 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.308 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.308 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.308 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.308 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.309 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.309 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.309 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.309 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.309 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.309 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.309 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.309 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.309 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.309 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.309 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.309 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.310 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.310 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.310 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.310 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.310 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.310 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.310 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.310 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.310 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.310 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.310 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.310 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2023-10-06T13:54:15.311 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.311 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.311 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.311 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.311 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.311 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_watch_notify_pp 2023-10-06T13:54:15.311 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.311 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.311 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.311 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.311 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.311 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.312 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.312 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.312 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.312 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2023-10-06T13:54:15.312 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.312 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.312 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.312 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.312 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_stat_pp 2023-10-06T13:54:15.312 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.312 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.313 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.313 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.313 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.313 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.313 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.313 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.313 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.313 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.313 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.313 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.313 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:15.313 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.314 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.314 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.314 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.314 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.314 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_service 2023-10-06T13:54:15.314 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92147 2023-10-06T13:54:15.314 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_service on pid 92147' 2023-10-06T13:54:15.314 INFO:tasks.workunit.client.0.smithi019.stdout:test api_service on pid 92147 2023-10-06T13:54:15.318 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=92147 2023-10-06T13:54:15.318 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.318 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:15.318 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:15.318 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.319 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_service_pp 2023-10-06T13:54:15.319 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_service_pp' 2023-10-06T13:54:15.319 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.319 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.319 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.319 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.319 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.319 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_service_pp 2023-10-06T13:54:15.319 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:15.319 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.319 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.319 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.320 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2023-10-06T13:54:15.320 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.320 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.320 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.320 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.320 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.320 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.320 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:15.320 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:15.320 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.320 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.321 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:15.321 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:15.321 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.321 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:15.321 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.321 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.321 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.321 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.321 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.321 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.321 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_service_pp 2023-10-06T13:54:15.321 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.321 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.322 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92160 2023-10-06T13:54:15.322 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.322 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_service_pp on pid 92160' 2023-10-06T13:54:15.322 INFO:tasks.workunit.client.0.smithi019.stdout:test api_service_pp on pid 92160 2023-10-06T13:54:15.326 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=92160 2023-10-06T13:54:15.327 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:15.327 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:15.327 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.327 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.327 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.327 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.327 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.327 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.327 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.327 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.328 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.328 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.328 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.328 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.328 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.328 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.328 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.328 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.328 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_c_write_operations 2023-10-06T13:54:15.328 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.328 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.328 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.329 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.329 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.329 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_c_write_operations' 2023-10-06T13:54:15.329 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.329 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.329 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.329 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.329 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.329 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.329 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.329 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.329 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.330 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.330 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2023-10-06T13:54:15.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.330 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.330 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.331 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.331 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/92111/exe 2023-10-06T13:54:15.331 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:15.331 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:15.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:15.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:15.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.331 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:15.332 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.332 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.332 INFO:tasks.workunit.client.0.smithi019.stdout::14.321+0000 7fa9bec0d780 10 monclient(hunting): authenticate will time out at 1083.507372s 2023-10-06T13:54:15.332 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.347+0000 7fa9b77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x563be692a9a0 0x563be6929a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:54:15.332 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.353+0000 7fa9b6ffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x563be6931c50 0x563be692fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:54:15.332 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.353+0000 7fa9b6ffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x563be6931c50 0x563be692fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:55276/0 (socket says 172.21.15.19:55276) 2023-10-06T13:54:15.332 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.353+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/3235738639 learned_addr learned my addr 172.21.15.19:0/3235738639 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:54:15.332 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.353+0000 7fa9b6ffd640 10 monclient(hunting): get_auth_request con 0x563be6931c50 auth_method 0 2023-10-06T13:54:15.332 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.353+0000 7fa9b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:54:15.333 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.353+0000 7fa9b6ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:54:15.333 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.353+0000 7fa9b6ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:54:15.333 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.355+0000 7fa9b77fe640 10 monclient(hunting): get_auth_request con 0x563be692a9a0 auth_method 0 2023-10-06T13:54:15.333 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.355+0000 7fa9b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:54:15.333 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.355+0000 7fa9b77fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:54:15.333 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.355+0000 7fa9b77fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:54:15.333 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.356+0000 7fa9b6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:54:15.333 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.356+0000 7fa9b6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:54:15.333 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.356+0000 7fa9b6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:54:15.333 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.357+0000 7fa9b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:54:15.333 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.357+0000 7fa9b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:54:15.333 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.357+0000 7fa9b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:54:15.334 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.359+0000 7fa9b6ffd640 10 monclient(hunting): handle_auth_done global_id 4349 payload 274 2023-10-06T13:54:15.334 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.359+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/3235738639 >> v2:172.21.15.83:3300/0 conn(0x563be692a9a0 msgr2=0x563be6929a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:54:15.334 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.359+0000 7fa9b6ffd640 1 --2- 172.21.15.19:0/3235738639 >> v2:172.21.15.83:3300/0 conn(0x563be692a9a0 0x563be6929a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.334 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.359+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/3235738639 >> v1:172.21.15.19:6789/0 conn(0x563be69302a0 legacy=0x563be69292f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T13:54:15.334 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.359+0000 7fa9b6ffd640 10 monclient: _finish_hunting 0 2023-10-06T13:54:15.334 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.359+0000 7fa9b6ffd640 1 monclient: found mon.noname-b 2023-10-06T13:54:15.336 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/92044/exe 2023-10-06T13:54:15.336 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.336 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.337 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.337 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.337 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.337 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.337 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.337 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.337 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.337 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.364 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_cmd_pp 2023-10-06T13:54:15.364 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_cmd_pp: /' 2023-10-06T13:54:15.364 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.364 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:15.364 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.364 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.365 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.365 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.365 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.365 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.365 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.365 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.365 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.365 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.365 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_c_write_operations 2023-10-06T13:54:15.366 INFO:tasks.workunit.client.0.smithi019.stdout: rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:54:15.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 10 monclient(hunting): picked mon.noname-c con 0x55d592aa5bc0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:54:15.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 10 monclient(hunting): start opening mon connection 2023-10-06T13:54:15.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5929d8e30 con 0x55d592aa2e10 2023-10-06T13:54:15.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 10 monclient(hunting): start opening mon connection 2023-10-06T13:54:15.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d592aa1670 con 0x55d5929d30c0 2023-10-06T13:54:15.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 10 monclient(hunting): start opening mon connection 2023-10-06T13:54:15.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d592aa9980 con 0x55d592aa5bc0 2023-10-06T13:54:15.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 10 monclient(hunting): _renew_subs 2023-10-06T13:54:15.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.315+0000 7f55c9d3c780 10 monclient(hunting): authenticate will time out at 1382.502076s 2023-10-06T13:54:15.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.348+0000 7f55c783b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5929d30c0 0x55d592aa0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:54:15.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.355+0000 7f55c783b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5929d30c0 0x55d592aa0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:59748/0 (socket says 172.21.15.19:59748) 2023-10-06T13:54:15.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.355+0000 7f55c783b640 1 -- 172.21.15.19:0/880871861 learned_addr learned my addr 172.21.15.19:0/880871861 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:54:15.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.355+0000 7f55c783b640 10 monclient(hunting): get_auth_request con 0x55d5929d30c0 auth_method 0 2023-10-06T13:54:15.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.355+0000 7f55c783b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:54:15.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.355+0000 7f55c783b640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:54:15.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.355+0000 7f55c783b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:54:15.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.364+0000 7f55c803c640 1 --2- 172.21.15.19:0/880871861 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x55d592aa5bc0 0x55d592b43600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:54:15.368 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.364+0000 7f55c803c640 10 monclient(hunting): get_auth_request con 0x55d592aa5bc0 auth_method 0 2023-10-06T13:54:15.368 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.364+0000 7f55c803c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:54:15.368 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.364+0000 7f55c803c640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:54:15.368 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.364+0000 7f55c803c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:54:15.368 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.364+0000 7f55c703a640 1 --2- 172.21.15.19:0/880871861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d592aa2e10 0x55d592aa1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:54:15.368 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.364+0000 7f55c703a640 10 monclient(hunting): get_auth_request con 0x55d592aa2e10 auth_method 0 2023-10-06T13:54:15.369 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:15.369 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_c_write_operations 2023-10-06T13:54:15.369 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_watch_notify: /' 2023-10-06T13:54:15.369 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92179 2023-10-06T13:54:15.369 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_c_write_operations on pid 92179' 2023-10-06T13:54:15.369 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2023-10-06T13:54:15.369 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio:test api_c_write_operations on pid 92179 2023-10-06T13:54:15.377 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=92179 2023-10-06T13:54:15.377 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:15.377 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:15.377 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:15.377 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:15.377 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.378 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:15.378 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.378 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:15.378 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:15.378 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.378 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:15.378 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.378 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:15.378 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.378 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.379 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.379 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.379 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.379 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.379 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.379 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_c_read_operations 2023-10-06T13:54:15.379 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_c_read_operations' 2023-10-06T13:54:15.379 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:15.379 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_c_read_operations 2023-10-06T13:54:15.379 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:15.380 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:15.380 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.380 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:15.380 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.380 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:15.380 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:15.380 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.380 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:15.380 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.380 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:15.380 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.380 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.381 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.381 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.381 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.381 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.381 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.381 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.381 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.381 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.381 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.381 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.381 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.381 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.381 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.382 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.382 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.382 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.382 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.382 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.382 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.382 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.382 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:15.382 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.382 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.382 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.382 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.383 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.383 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.383 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:15.383 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.383 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.383 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.383 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.383 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.383 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.383 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.383 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.383 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:15.384 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.384 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.384 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.384 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.384 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.384 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.384 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:15.384 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.384 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.384 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.384 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.384 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.385 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.385 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.385 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.385 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.385 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.385 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.385 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.385 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:15.385 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.385 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:15.385 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2023-10-06T13:54:15.385 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_c_read_operations 2023-10-06T13:54:15.386 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:15.386 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92198 2023-10-06T13:54:15.386 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_c_read_operations on pid 92198' 2023-10-06T13:54:15.386 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:15.386 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:15.386 INFO:tasks.workunit.client.0.smithi019.stdout:test api_c_read_operations on pid 92198 2023-10-06T13:54:15.392 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=92198 2023-10-06T13:54:15.392 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.392 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:15.392 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.392 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:15.393 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:15.393 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:15.393 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.393 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:15.393 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.393 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:15.393 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.393 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.393 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.393 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.393 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.394 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.394 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.394 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:15.394 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_watch_notify 2023-10-06T13:54:15.394 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s api_cls_remote_reads 2023-10-06T13:54:15.394 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2023-10-06T13:54:15.394 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.394 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.394 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' api_cls_remote_reads' 2023-10-06T13:54:15.394 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.394 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.394 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.395 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.395 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.395 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.395 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.395 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.395 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.395 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/92082/exe 2023-10-06T13:54:15.395 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.395 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.395 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.395 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.395 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.396 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.396 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.396 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.396 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.396 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_watch_notify.log 2023-10-06T13:54:15.396 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.396 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.396 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.396 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.396 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.396 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.396 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.396 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.397 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:15.397 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:15.397 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.397 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.397 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.397 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:15.397 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:15.397 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.397 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:15.397 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_cmd.log 2023-10-06T13:54:15.397 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.397 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_cmd 2023-10-06T13:54:15.398 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.398 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.398 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.398 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo api_cls_remote_reads 2023-10-06T13:54:15.398 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.398 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.398 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.398 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.398 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.398 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_cmd: /' 2023-10-06T13:54:15.399 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.359+0000 7fa9b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:54:15.399 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.359+0000 7fa9b6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T13:54:15.399 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.359+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/3235738639 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563be5d014e0 con 0x563be6931c50 2023-10-06T13:54:15.399 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.374+0000 7fa9b6ffd640 1 --2- 172.21.15.19:0/3235738639 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x563be6931c50 0x563be692fd60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800a5a0 tx=0x7fa9a801cb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e41cffa6c74b531 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:54:15.399 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9b5ffb640 1 -- 172.21.15.19:0/3235738639 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a804abf0 con 0x563be6931c50 2023-10-06T13:54:15.399 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9b5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:54:15.399 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9b5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:54:15.399 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9b5ffb640 10 monclient: dump: 2023-10-06T13:54:15.399 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: epoch 1 2023-10-06T13:54:15.399 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:54:15.399 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.400 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.400 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: min_mon_release 18 (reef) 2023-10-06T13:54:15.400 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: election_strategy: 1 2023-10-06T13:54:15.400 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:54:15.400 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:54:15.400 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:54:15.400 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:15.400 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9b5ffb640 10 monclient: _finish_auth 0 2023-10-06T13:54:15.400 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9b5ffb640 10 monclient: _check_auth_tickets 2023-10-06T13:54:15.400 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9b5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:15.400 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9b5ffb640 1 -- 172.21.15.19:0/3235738639 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a804ad50 con 0x563be6931c50 2023-10-06T13:54:15.400 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9b5ffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:54:15.401 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9b5ffb640 1 -- 172.21.15.19:0/3235738639 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8052660 con 0x563be6931c50 2023-10-06T13:54:15.401 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9b5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:54:15.401 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9b5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T13:54:15.401 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9b5ffb640 10 monclient: dump: 2023-10-06T13:54:15.401 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: epoch 1 2023-10-06T13:54:15.401 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:54:15.401 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.401 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.401 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: min_mon_release 18 (reef) 2023-10-06T13:54:15.401 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: election_strategy: 1 2023-10-06T13:54:15.401 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:54:15.402 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:54:15.402 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:54:15.402 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:15.402 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9bec0d780 5 monclient: authenticate success, global_id 4349 2023-10-06T13:54:15.404 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:15.404 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.404 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.404 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.404 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.405 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.405 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.405 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.405 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.405 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.405 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.405 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.405 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.405 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.405 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.405 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.405 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.406 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.406 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.406 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.406 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.406 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.406 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.406 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:15.406 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.406 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.406 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.406 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.406 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.407 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9bec0d78 2023-10-06T13:54:14.364+0000 7f55c703a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:54:15.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.364+0000 7f55c703a640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:54:15.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.364+0000 7f55c703a640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:54:15.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.367+0000 7f55c783b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:54:15.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.367+0000 7f55c783b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:54:15.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.367+0000 7f55c783b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:54:15.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.368+0000 7f55c703a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:54:15.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.368+0000 7f55c703a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:54:15.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.368+0000 7f55c703a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:54:15.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.370+0000 7f55c783b640 10 monclient(hunting): handle_auth_done global_id 4453 payload 824 2023-10-06T13:54:15.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.370+0000 7f55c783b640 1 -- 172.21.15.19:0/880871861 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x55d592aa5bc0 msgr2=0x55d592b43600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:54:15.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.370+0000 7f55c783b640 1 --2- 172.21.15.19:0/880871861 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x55d592aa5bc0 0x55d592b43600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.370+0000 7f55c783b640 1 -- 172.21.15.19:0/880871861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d592aa2e10 msgr2=0x55d592aa1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:54:15.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.370+0000 7f55c783b640 1 --2- 172.21.15.19:0/880871861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d592aa2e10 0x55d592aa1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.370+0000 7f55c783b640 10 monclient: _finish_hunting 0 2023-10-06T13:54:15.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.370+0000 7f55c783b640 1 monclient: found mon.noname-b 2023-10-06T13:54:15.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.370+0000 7f55c783b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:54:15.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.370+0000 7f55c783b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T13:54:15.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.370+0000 7f55c783b640 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8005d70 con 0x55d5929d30c0 2023-10-06T13:54:15.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.382+0000 7f55c783b640 1 --2- 172.21.15.19:0/880871861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5929d30c0 0x55d592aa0bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f55b8013cb0 tx=0x7f55b8002fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:54:15.409 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.382+0000 7f55c5036640 10 client.?.objecter ms_handle_connect 0x55d5929d30c0 2023-10-06T13:54:15.409 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.382+0000 7f55c5036640 10 client.?.objecter resend_mon_ops 2023-10-06T13:54:15.409 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.394+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8013e90 con 0x55d5929d30c0 2023-10-06T13:54:15.409 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.394+0000 7f55c5036640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:54:15.409 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.394+0000 7f55c5036640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T13:54:15.411 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.411 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:15.411 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:15.411 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.411 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:15.411 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.411 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:15.411 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:15.412 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.412 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:15.412 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.412 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:15.412 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.412 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.412 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.412 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.412 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.412 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.412 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.412 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=api_cls_remote_reads 2023-10-06T13:54:15.413 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92266 2023-10-06T13:54:15.413 INFO:tasks.workunit.client.0.smithi019.stdout:test api_cls_remote_reads on pid 92266 2023-10-06T13:54:15.413 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test api_cls_remote_reads on pid 92266' 2023-10-06T13:54:15.414 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=92266 2023-10-06T13:54:15.414 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:15.414 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:15.414 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2023-10-06T13:54:15.414 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:15.414 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:15.414 INFO:tasks.workunit.client.0.smithi019.stdout:0 10 monclient: get_monmap_and_config success 2023-10-06T13:54:15.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.391+0000 7fa9bec0d780 4 set_mon_vals no callback set 2023-10-06T13:54:15.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.392+0000 7fa9bec0d780 10 monclient: shutdown 2023-10-06T13:54:15.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.392+0000 7fa9bec0d780 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:54:15.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.392+0000 7fa9bec0d780 1 -- 172.21.15.19:0/3235738639 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x563be6931c50 msgr2=0x563be692fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:54:15.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.392+0000 7fa9bec0d780 1 --2- 172.21.15.19:0/3235738639 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x563be6931c50 0x563be692fd60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800a5a0 tx=0x7fa9a801cb70 comp rx=0 tx=0).stop 2023-10-06T13:54:15.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.397+0000 7fa9bec0d780 1 -- 172.21.15.19:0/3235738639 shutdown_connections 2023-10-06T13:54:15.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.397+0000 7fa9bec0d780 1 --2- 172.21.15.19:0/3235738639 >> v2:172.21.15.83:3300/0 conn(0x563be692a9a0 0x563be6929a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.397+0000 7fa9bec0d780 1 --2- 172.21.15.19:0/3235738639 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x563be6931c50 0x563be692fd60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.397+0000 7fa9bec0d780 1 -- 172.21.15.19:0/3235738639 >> 172.21.15.19:0/3235738639 conn(0x563be6928310 msgr2=0x563be68edb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:54:15.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.397+0000 7fa9bec0d780 1 -- 172.21.15.19:0/3235738639 shutdown_connections 2023-10-06T13:54:15.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.397+0000 7fa9bec0d780 1 -- 172.21.15.19:0/3235738639 wait complete. 2023-10-06T13:54:15.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.398+0000 7fa9bec0d780 -1 asok(0x563be5d83840) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-client.admin.91880.asok': (17) File exists 2023-10-06T13:54:15.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.398+0000 7fa9bec0d780 10 monclient: build_initial_monmap 2023-10-06T13:54:15.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.398+0000 7fa9bec0d780 10 monclient: monmap: 2023-10-06T13:54:15.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: epoch 0 2023-10-06T13:54:15.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: fsid 00000000-0000-0000-0000-000000000000 2023-10-06T13:54:15.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: last_changed 0.000000 2023-10-06T13:54:15.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: created 0.000000 2023-10-06T13:54:15.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: min_mon_release 0 (unknown) 2023-10-06T13:54:15.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: election_strategy: 1 2023-10-06T13:54:15.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T13:54:15.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T13:54:15.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T13:54:15.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:15.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.399+0000 7fa9bec0d780 1 librados: starting msgr at 2023-10-06T13:54:15.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.399+0000 7fa9bec0d780 1 librados: starting objecter 2023-10-06T13:54:15.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.399+0000 7fa9bec0d780 1 Processor -- start 2023-10-06T13:54:15.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.399+0000 7fa9bec0d780 1 -- start start 2023-10-06T13:54:15.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.399+0000 7fa9bec0d780 1 librados: setting wanted keys 2023-10-06T13:54:15.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.399+0000 7fa9bec0d780 1 librados: calling monclient init 2023-10-06T13:54:15.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.399+0000 7fa9bec0d780 10 monclient: init 2023-10-06T13:54:15.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 10 monclient: _reopen_session rank -1 2023-10-06T13:54:15.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T13:54:15.421 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:15.421 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:15.421 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.421 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:15.421 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s list_parallel 2023-10-06T13:54:15.421 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' list_parallel' 2023-10-06T13:54:15.422 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.422 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.422 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.422 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.422 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.422 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.422 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.422 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.422 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:15.422 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.422 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.423 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.423 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.423 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.423 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.423 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.423 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.423 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.423 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo list_parallel 2023-10-06T13:54:15.423 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.423 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.423 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.423 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.424 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.424 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.424 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.424 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.424 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.424 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.424 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.424 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.424 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.424 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.424 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.424 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.424 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.425 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.425 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.425 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=list_parallel 2023-10-06T13:54:15.425 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.425 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:15.425 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.425 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92288 2023-10-06T13:54:15.425 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563be692a9a0 0test list_parallel on pid 92288 2023-10-06T13:54:15.437 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test list_parallel on pid 92288' 2023-10-06T13:54:15.437 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=92288 2023-10-06T13:54:15.437 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.438 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:15.438 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:15.438 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:15.438 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2023-10-06T13:54:15.438 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:15.438 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.438 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.438 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.438 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.438 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.438 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.439 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:15.439 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.439 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:15.439 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:15.439 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.439 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:15.439 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.439 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:15.439 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.439 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.439 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.439 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.440 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.440 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.440 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.440 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:15.440 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:15.440 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:15.440 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:15.440 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.440 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:15.440 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.440 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.440 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.441 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.441 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.441 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.441 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.441 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.441 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.441 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.441 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.441 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.441 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.441 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.442 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.442 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.442 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.442 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.442 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.442 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.442 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.442 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.442 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.442 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.442 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.442 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.443 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s open_pools_parallel 2023-10-06T13:54:15.443 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' open_pools_parallel' 2023-10-06T13:54:15.443 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.443 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:15.443 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:15.443 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:15.443 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:15.443 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.443 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:15.443 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.443 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:15.443 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:15.444 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.444 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:15.444 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.444 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:15.444 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.444 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.444 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.444 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.444 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.444 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.444 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.444 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.445 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.445 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.445 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.445 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.445 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.445 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.445 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.445 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.445 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.445 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.445 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.445 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.446 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.446 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.446 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo open_pools_parallel 2023-10-06T13:54:15.446 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.446 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.446 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.446 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.446 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.446 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.446 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.446 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.446 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.447 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.447 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.447 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.447 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.447 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.447 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.447 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.447 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.447 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.447 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.447 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.447 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.448 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.448 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.448 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.448 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.448 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.448 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.448 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.448 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.448 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.448 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.448 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.448 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.449 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.449 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.449 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.449 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.449 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.449 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.449 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.449 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.449 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.449 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.449 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/92147/exe 2023-10-06T13:54:15.449 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.450 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.450 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.450 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.450 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.450 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.450 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.450 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.450 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.450 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.450 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.450 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.450 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.451 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.451 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.451 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.451 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.451 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.451 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=open_pools_parallel 2023-10-06T13:54:15.451 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92313 2023-10-06T13:54:15.451 INFO:tasks.workunit.client.0.smithi019.stdout:test open_pools_parallel on pid 92313 2023-10-06T13:54:15.467 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test open_pools_parallel on pid 92313' 2023-10-06T13:54:15.467 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=92313 2023-10-06T13:54:15.467 INFO:tasks.workunit.client.0.smithi019.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-06T13:54:15.467 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:15.467 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2023-10-06T13:54:15.467 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.467 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.468 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.468 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.468 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_service 2023-10-06T13:54:15.468 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.468 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.468 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.468 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.468 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.468 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.468 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.468 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.468 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.469 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.469 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_service: /' 2023-10-06T13:54:15.469 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.469 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.469 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.469 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.469 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.469 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.469 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.469 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.469 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.470 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.470 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.470 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.470 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.470 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.470 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.470 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.470 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.470 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.470 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.470 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.470 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.471 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.471 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_service.log 2023-10-06T13:54:15.471 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:15.471 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:15.471 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:15.471 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:15.471 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.471 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:15.471 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/92160/exe 2023-10-06T13:54:15.471 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.471 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.471 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.472 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.472 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.472 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.472 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:15.472 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.472 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.472 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.472 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:15.472 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:15.472 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.472 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.472 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.473 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.473 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.473 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.473 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.473 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.473 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:15.473 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.473 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.473 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.473 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.473 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.473 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.474 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.474 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:15.474 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:15.474 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.474 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:15.474 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.474 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:15.474 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.474 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.474 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.474 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.474 INFO:tasks.workunit.client.0.smithi019.stderr:++ printf %25s delete_pools_parallel 2023-10-06T13:54:15.475 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.475 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.475 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.475 INFO:tasks.workunit.client.0.smithi019.stderr:+ r=' delete_pools_parallel' 2023-10-06T13:54:15.475 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/92179/exe 2023-10-06T13:54:15.475 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.475 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.475 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.475 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.475 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.475 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.475 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.476 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.476 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.476 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.476 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.476 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.476 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.476 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.476 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.476 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.476 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.476 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.476 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.477 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.477 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.477 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.477 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.477 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.477 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.477 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.477 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.477 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.477 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.477 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.477 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.478 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.478 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.478 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.478 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_service_pp 2023-10-06T13:54:15.478 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.478 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.478 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.478 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.478 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.478 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.478 INFO:tasks.workunit.client.0.smithi019.stderr:++ echo delete_pools_parallel 2023-10-06T13:54:15.478 INFO:tasks.workunit.client.0.smithi019.stderr:++ awk '{print $1}' 2023-10-06T13:54:15.479 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.479 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.479 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.479 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.479 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.479 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.479 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.479 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.479 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.479 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.479 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_c_write_operations 2023-10-06T13:54:15.479 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2023-10-06T13:54:15.480 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_service_pp.log 2023-10-06T13:54:15.480 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.480 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_c_write_operations: /' 2023-10-06T13:54:15.480 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.480 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.480 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.480 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.480 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.480 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.480 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.480 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.480 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.480 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.481 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.481 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.481 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.481 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.481 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.481 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.481 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.481 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.481 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.481 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.481 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.481 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.482 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.482 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.482 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.482 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.482 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.482 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.482 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.482 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.482 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.482 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.482 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.482 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.483 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.483 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.483 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.483 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.483 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.483 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.483 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.483 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.483 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.483 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.483 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.484 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_service_pp: /' 2023-10-06T13:54:15.484 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.484 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.484 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.484 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.484 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.484 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.484 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.484 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.484 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.484 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.484 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.485 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.485 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.485 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.485 INFO:tasks.workunit.client.0.smithi019.stderr:+ ff=delete_pools_parallel 2023-10-06T13:54:15.485 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.485 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.485 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92354 2023-10-06T13:54:15.485 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.485 INFO:tasks.workunit.client.0.smithi019.stdout:test delete_pools_parallel on pid 92354 2023-10-06T13:54:15.493 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'test delete_pools_parallel on pid 92354' 2023-10-06T13:54:15.493 INFO:tasks.workunit.client.0.smithi019.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2023-10-06T13:54:15.493 INFO:tasks.workunit.client.0.smithi019.stderr:+ pids[$f]=92354 2023-10-06T13:54:15.493 INFO:tasks.workunit.client.0.smithi019.stderr:+ ret=0 2023-10-06T13:54:15.493 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' 1 -eq 1 ']' 2023-10-06T13:54:15.493 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T13:54:15.493 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91777 2023-10-06T13:54:15.493 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91777 2023-10-06T13:54:15.493 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.493 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.493 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -f /etc/bashrc ']' 2023-10-06T13:54:15.494 INFO:tasks.workunit.client.0.smithi019.stderr:+ . /etc/bashrc 2023-10-06T13:54:15.494 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -z '' ']' 2023-10-06T13:54:15.494 INFO:tasks.workunit.client.0.smithi019.stderr:++ BASHRCSOURCED=Y 2023-10-06T13:54:15.494 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.494 INFO:tasks.workunit.client.0.smithi019.stderr:++ shopt -q login_shell 2023-10-06T13:54:15.494 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.494 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.494 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.494 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.494 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.494 INFO:tasks.workunit.client.0.smithi019.stderr:+++ umask 2023-10-06T13:54:15.495 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' 0022 -eq 0 ']' 2023-10-06T13:54:15.495 INFO:tasks.workunit.client.0.smithi019.stderr:++ SHELL=/bin/bash 2023-10-06T13:54:15.495 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.495 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.495 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-06T13:54:15.495 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.495 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-06T13:54:15.495 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-06T13:54:15.495 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.495 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-06T13:54:15.495 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.495 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-06T13:54:15.496 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.496 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.496 INFO:tasks.workunit.client.0.smithi019.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-06T13:54:15.496 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.496 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-06T13:54:15.496 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.496 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-06T13:54:15.496 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.496 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.496 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.496 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.496 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.497 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.497 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.497 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.497 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.497 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.497 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.497 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.497 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.497 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.497 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.497 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.497 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.498 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.498 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.498 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.498 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.498 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.498 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.498 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.498 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.498 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.498 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.498 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.499 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.499 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.499 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.499 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.499 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.499 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.499 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.499 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.499 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.499 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.499 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.499 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.500 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.500 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.500 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.500 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.500 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.500 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.500 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.500 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.500 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.500 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.500 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.501 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.501 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.501 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.501 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.501 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.501 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.501 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.501 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.501 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.394+0000 7f55c5036640 10 monclient: dump: 2023-10-06T13:54:15.502 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: epoch 1 2023-10-06T13:54:15.502 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:54:15.502 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.502 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.502 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: min_mon_release 18 (reef) 2023-10-06T13:54:15.502 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: election_strategy: 1 2023-10-06T13:54:15.502 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:54:15.502 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:54:15.502 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:54:15.502 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.502 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c5036640 10 monclient: _finish_auth 0 2023-10-06T13:54:15.502 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c5036640 10 monclient: _check_auth_tickets 2023-10-06T13:54:15.503 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c5036640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:15.503 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b8005510 con 0x55d5929d30c0 2023-10-06T13:54:15.503 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c5036640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:54:15.503 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8008a60 con 0x55d5929d30c0 2023-10-06T13:54:15.503 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c5036640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:54:15.503 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c5036640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T13:54:15.503 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c5036640 10 monclient: dump: 2023-10-06T13:54:15.503 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: epoch 1 2023-10-06T13:54:15.503 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:54:15.503 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.503 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.504 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: min_mon_release 18 (reef) 2023-10-06T13:54:15.504 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: election_strategy: 1 2023-10-06T13:54:15.504 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:54:15.504 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:54:15.504 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:54:15.504 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.504 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c9d3c780 5 monclient: authenticate success, global_id 4453 2023-10-06T13:54:15.504 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c9d3c780 10 monclient: _renew_subs 2023-10-06T13:54:15.504 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c9d3c780 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:54:15.504 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d592937340 con 0x55d5929d30c0 2023-10-06T13:54:15.723 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c9d3c780 10 client.4453.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:54:15.723 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c9d3c780 10 monclient: _renew_subs 2023-10-06T13:54:15.723 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.395+0000 7f55c6038640 4 set_mon_vals no callback set 2023-10-06T13:54:15.724 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.396+0000 7f55c9d3c780 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:54:15.724 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.396+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d59292a780 con 0x55d5929d30c0 2023-10-06T13:54:15.724 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.396+0000 7f55c9d3c780 1 librados: init done 2023-10-06T13:54:15.724 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.396+0000 7f55c9d3c780 10 librados: wait_for_osdmap waiting 2023-10-06T13:54:15.726 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.726 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.726 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.727 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.727 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.727 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.727 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.727 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.727 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.727 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.727 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.727 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.727 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.727 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.727 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.728 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.728 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/92198/exe 2023-10-06T13:54:15.728 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.444+0000 7f55c5x563be692e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:54:15.728 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 10 monclient(hunting): picked mon.noname-a con 0x563be692a9a0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T13:54:15.728 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563be69302a0 0x563be68ea900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:54:15.728 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 10 monclient(hunting): picked mon.noname-b con 0x563be69302a0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T13:54:15.728 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x563be6931c50 0x563be69b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:54:15.728 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 10 monclient(hunting): picked mon.noname-c con 0x563be6931c50 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T13:54:15.728 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 10 monclient(hunting): start opening mon connection 2023-10-06T13:54:15.729 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be5cd7f20 con 0x563be692a9a0 2023-10-06T13:54:15.729 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 10 monclient(hunting): start opening mon connection 2023-10-06T13:54:15.729 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be5cbb420 con 0x563be69302a0 2023-10-06T13:54:15.729 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 10 monclient(hunting): start opening mon connection 2023-10-06T13:54:15.729 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x563be68eae40 con 0x563be6931c50 2023-10-06T13:54:15.729 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 10 monclient(hunting): _renew_subs 2023-10-06T13:54:15.729 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.400+0000 7fa9bec0d780 10 monclient(hunting): authenticate will time out at 1083.587161s 2023-10-06T13:54:15.729 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.425+0000 7fa9b67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563be69302a0 0x563be68ea900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:54:15.729 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.429+0000 7fa9b67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563be69302a0 0x563be68ea900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:59834/0 (socket says 172.21.15.19:59834) 2023-10-06T13:54:15.729 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.429+0000 7fa9b67fc640 1 -- 172.21.15.19:0/2639474284 learned_addr learned my addr 172.21.15.19:0/2639474284 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T13:54:15.729 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.429+0000 7fa9b67fc640 10 monclient(hunting): get_auth_request con 0x563be69302a0 auth_method 0 2023-10-06T13:54:15.730 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.429+0000 7fa9b67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:54:15.730 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.429+0000 7fa9b67fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:54:15.730 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.429+0000 7fa9b67fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:54:15.730 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.445+0000 7fa9b6ffd640 1 --2- 172.21.15.19:0/2639474284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563be692a9a0 0x563be692e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:54:15.737 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.737 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.737 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.737 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.737 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.737 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.737 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/92266/exe 2023-10-06T13:54:15.737 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.737 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.738 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.738 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.738 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.738 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.738 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.738 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.738 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.738 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.738 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.738 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.738 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.739 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.739 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.739 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.739 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.739 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.739 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.739 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.739 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.739 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.739 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_c_read_operations: /' 2023-10-06T13:54:15.739 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.739 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.740 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.740 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.740 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.740 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.740 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.740 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.740 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.740 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.740 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.740 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.740 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.740 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.740 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.741 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.741 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.741 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2023-10-06T13:54:15.741 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2023-10-06T13:54:15.741 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'grep=grep --color=auto' 2023-10-06T13:54:15.741 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-06T13:54:15.741 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-06T13:54:15.741 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.741 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-06T13:54:15.741 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.741 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.741 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.742 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.742 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.742 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.742 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.742 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.742 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.742 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.742 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.742 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.742 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.742 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.742 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.743 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.743 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.743 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.743 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.743 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.743 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.743 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.743 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorls.sh 2023-10-06T13:54:15.743 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' '!' -t 0 ']' 2023-10-06T13:54:15.743 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_c_read_operations 2023-10-06T13:54:15.743 INFO:tasks.workunit.client.0.smithi019.stderr:+++ return 2023-10-06T13:54:15.743 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.744 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-06T13:54:15.744 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.744 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-06T13:54:15.744 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export S_COLORS=auto 2023-10-06T13:54:15.744 INFO:tasks.workunit.client.0.smithi019.stderr:+++ S_COLORS=auto 2023-10-06T13:54:15.744 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.744 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-06T13:54:15.744 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.744 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-06T13:54:15.744 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.744 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/92288/exe 2023-10-06T13:54:15.744 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.745 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.745 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.745 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.745 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.745 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.745 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.745 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.745 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.745 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.745 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.745 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_api_cls_remote_reads 2023-10-06T13:54:15.745 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_list_parallel 2023-10-06T13:54:15.745 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2023-10-06T13:54:15.746 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-06T13:54:15.746 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-06T13:54:15.746 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-06T13:54:15.746 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.746 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-06T13:54:15.746 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.746 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-06T13:54:15.746 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-06T13:54:15.746 INFO:tasks.workunit.client.0.smithi019.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-06T13:54:15.746 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ list_parallel: /' 2023-10-06T13:54:15.746 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.747 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-06T13:54:15.747 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-06T13:54:15.747 INFO:tasks.workunit.client.0.smithi019.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-06T13:54:15.747 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.747 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-06T13:54:15.747 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.747 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-06T13:54:15.747 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-06T13:54:15.747 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.747 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-06T13:54:15.747 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.747 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/gawk.sh 2023-10-06T13:54:15.747 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.748 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-06T13:54:15.748 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.748 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/lang.sh 2023-10-06T13:54:15.748 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup 2023-10-06T13:54:15.748 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/locale 2023-10-06T13:54:15.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.448+0000 7fa9b6ffd640 10 monclient(hunting): get_auth_request con 0x563be692a9a0 auth_method 0 2023-10-06T13:54:15.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.448+0000 7fa9b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:54:15.748 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.448+0000 7fa9b6ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:54:15.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.448+0000 7fa9b6ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:54:15.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.450+0000 7fa9b77fe640 1 --2- 172.21.15.19:0/2639474284 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x563be6931c50 0x563be69b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:54:15.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.452+0000 7fa9b77fe640 10 monclient(hunting): get_auth_request con 0x563be6931c50 auth_method 0 2023-10-06T13:54:15.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.452+0000 7fa9b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T13:54:15.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.452+0000 7fa9b77fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T13:54:15.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.452+0000 7fa9b77fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T13:54:15.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.453+0000 7fa9b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:54:15.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.453+0000 7fa9b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:54:15.749 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.453+0000 7fa9b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:54:15.753 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.466+0000 7fa9b6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T13:54:15.753 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.466+0000 7fa9b6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T13:54:15.753 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.466+0000 7fa9b6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T13:54:15.753 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.471+0000 7fa9b6ffd640 10 monclient(hunting): handle_auth_done global_id 4452 payload 824 2023-10-06T13:54:15.753 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.471+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/2639474284 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x563be6931c50 msgr2=0x563be69b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:54:15.754 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.471+0000 7fa9b6ffd640 1 --2- 172.21.15.19:0/2639474284 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x563be6931c50 0x563be69b5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.754 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.471+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/2639474284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563be69302a0 msgr2=0x563be68ea900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:54:15.754 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.471+0000 7fa9b6ffd640 1 --2- 172.21.15.19:0/2639474284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563be69302a0 0x563be68ea900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.754 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.471+0000 7fa9b6ffd640 10 monclient: _finish_hunting 0 2023-10-06T13:54:15.754 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.471+0000 7fa9b6ffd640 1 monclient: found mon.noname-a 2023-10-06T13:54:15.754 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.471+0000 7fa9b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:54:15.754 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.471+0000 7fa9b6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T13:54:15.754 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.471+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/2639474284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a8005de0 con 0x563be692a9a0 2023-10-06T13:54:15.761 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.761 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.761 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.762 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.762 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.762 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.762 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.762 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-06T13:54:15.762 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.762 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-06T13:54:15.762 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-06T13:54:15.762 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.762 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_list_parallel.log 2023-10-06T13:54:15.762 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.763 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.763 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.763 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.763 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.763 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.763 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.763 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.763 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.763 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.763 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.763 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.763 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.764 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.764 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.764 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.764 INFO:tasks.workunit.client.0.smithi019.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-06T13:54:15.764 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.764 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.764 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.764 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.764 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/92354/exe 2023-10-06T13:54:15.764 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.764 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.764 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.765 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.765 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.765 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.765 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.765 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.765 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.765 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.765 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.765 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2023-10-06T13:54:15.765 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_delete_pools_parallel 2023-10-06T13:54:15.765 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ delete_pools_parallel: /' 2023-10-06T13:54:15.765 INFO:tasks.workunit.client.0.smithi019.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-06T13:54:15.765 INFO:tasks.workunit.client.0.smithi019.stderr:++++ export LANG=en_US.UTF-8 2023-10-06T13:54:15.766 INFO:tasks.workunit.client.0.smithi019.stderr:++++ LANG=en_US.UTF-8 2023-10-06T13:54:15.766 INFO:tasks.workunit.client.0.smithi019.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-06T13:54:15.766 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-06T13:54:15.766 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.766 INFO:tasks.workunit.client.0.smithi019.stderr:+++ LANG=en_US.UTF-8 2023-10-06T13:54:15.766 INFO:tasks.workunit.client.0.smithi019.stderr:+++ unset LANG_backup config 2023-10-06T13:54:15.766 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n '' ']' 2023-10-06T13:54:15.766 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-06T13:54:15.766 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' dumb = linux ']' 2023-10-06T13:54:15.766 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.766 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-06T13:54:15.766 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.767 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/less.sh 2023-10-06T13:54:15.767 INFO:tasks.workunit.client.0.smithi019.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-06T13:54:15.767 INFO:tasks.workunit.client.0.smithi019.stderr:++ for i in /etc/profile.d/*.sh 2023-10-06T13:54:15.767 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-06T13:54:15.767 INFO:tasks.workunit.client.0.smithi019.stderr:++ '[' '' ']' 2023-10-06T13:54:15.767 INFO:tasks.workunit.client.0.smithi019.stderr:++ . /etc/profile.d/which2.sh 2023-10-06T13:54:15.767 INFO:tasks.workunit.client.0.smithi019.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-06T13:54:15.767 INFO:tasks.workunit.client.0.smithi019.stderr:+++++ readlink /proc/92313/exe 2023-10-06T13:54:15.767 INFO:tasks.workunit.client.0.smithi019.stderr:++++ basename /usr/bin/bash 2023-10-06T13:54:15.767 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_declare='declare -f' 2023-10-06T13:54:15.769 INFO:tasks.workunit.client.0.smithi019.stderr:+++ which_opt=-f 2023-10-06T13:54:15.769 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export which_declare 2023-10-06T13:54:15.769 INFO:tasks.workunit.client.0.smithi019.stderr:+++ export -f which 2023-10-06T13:54:15.770 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset i 2023-10-06T13:54:15.770 INFO:tasks.workunit.client.0.smithi019.stderr:++ unset -f pathmunge 2023-10-06T13:54:15.770 INFO:tasks.workunit.client.0.smithi019.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-06T13:54:15.770 INFO:tasks.workunit.client.0.smithi019.stderr:+ export PATH 2023-10-06T13:54:15.770 INFO:tasks.workunit.client.0.smithi019.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-06T13:54:15.770 INFO:tasks.workunit.client.0.smithi019.stderr:+ unset rc 2023-10-06T13:54:15.770 INFO:tasks.workunit.client.0.smithi019.stderr:+ ceph_test_rados_open_pools_parallel 2023-10-06T13:54:15.770 INFO:tasks.workunit.client.0.smithi019.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2023-10-06T13:54:15.770 INFO:tasks.workunit.client.0.smithi019.stderr:+ sed 's/^/ open_pools_parallel: /' 2023-10-06T13:54:15.771 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.476+0000 7fa9b6ffd640 1 --2- 172.21.15.19:0/2639474284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563be692a9a0 0x563be692e970 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8051000 tx=0x7fa9a8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:54:15.771 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.477+0000 7fa9a77fe640 10 client.?.objecter ms_handle_connect 0x563be692a9a0 2023-10-06T13:54:15.771 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.477+0000 7fa9a77fe640 10 client.?.objecter resend_mon_ops 2023-10-06T13:54:15.771 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9a77fe640 1 -- 172.21.15.19:0/2639474284 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8052810 con 0x563be692a9a0 2023-10-06T13:54:15.771 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:54:15.771 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9a77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T13:54:15.771 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9a77fe640 10 monclient: dump: 2023-10-06T13:54:15.771 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: epoch 1 2023-10-06T13:54:15.771 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:54:15.771 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.771 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.772 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: min_mon_release 18 (reef) 2023-10-06T13:54:15.772 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: election_strategy: 1 2023-10-06T13:54:15.772 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:54:15.772 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:54:15.772 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:54:15.772 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:15.772 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9a77fe640 10 monclient: _finish_auth 0 2023-10-06T13:54:15.772 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9a77fe640 10 monclient: _check_auth_tickets 2023-10-06T13:54:15.772 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:15.772 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9a77fe640 1 -- 172.21.15.19:0/2639474284 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a80529b0 con 0x563be692a9a0 2023-10-06T13:54:15.772 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9a77fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T13:54:15.772 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9a77fe640 1 -- 172.21.15.19:0/2639474284 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8052d60 con 0x563be692a9a0 2023-10-06T13:54:15.773 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T13:54:15.773 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9a77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T13:54:15.773 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9a77fe640 10 monclient: dump: 2023-10-06T13:54:15.773 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: epoch 1 2023-10-06T13:54:15.773 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:54:15.773 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.773 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: created 2023-10-06T13:51:23.675166+0000 2023-10-06T13:54:15.773 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: min_mon_release 18 (reef) 2023-10-06T13:54:15.773 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: election_strategy: 1 2023-10-06T13:54:15.773 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T13:54:15.773 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T13:54:15.773 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T13:54:15.773 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:15.774 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9b57fa640 4 set_mon_vals no callback set 2023-10-06T13:54:15.774 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9bec0d780 5 monclient: authenticate success, global_id 4452 2023-10-06T13:54:15.774 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9bec0d780 10 monclient: _renew_subs 2023-10-06T13:54:15.774 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9bec0d780 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:54:15.774 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.510+0000 7fa9bec0d780 1 -- 172.21.15.19:0/2639474284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563be5ce7340 con 0x563be692a9a0 2023-10-06T13:54:15.774 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.511+0000 7fa9bec0d780 10 client.4452.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:54:15.774 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.511+0000 7fa9bec0d780 10 monclient: _renew_subs 2023-10-06T13:54:15.774 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.511+0000 7fa9bec0d780 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T13:54:15.774 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.511+0000 7fa9bec0d780 1 -- 172.21.15.19:0/2639474284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563be5cda780 con 0x563be692a9a0 2023-10-06T13:54:15.774 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.511+0000 7fa9bec0d780 1 librados: init done 2023-10-06T13:54:15.774 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.511+0000 7fa9bec0d780 10 librados: watch_flush enter 2023-10-06T13:54:15.774 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.511+0000 7fa9bec0d780 10 librados: watch_flush exit 2023-10-06T13:54:15.775 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.511+0000 7fa9bec0d780 20 client.4452.objecter shutdown clearing up homeless session... 2023-10-06T13:54:15.775 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.511+0000 7fa9bec0d780 10 client.4452.objecter successfully canceled tick 2023-10-06T13:54:15.775 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.511+0000 7fa9bec0d780 10 monclient: shutdown 2023-10-06T13:54:15.775 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.511+0000 7fa9bec0d780 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:54:15.775 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.511+0000 7fa9bec0d780 1 -- 172.21.15.19:0/2639474284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563be692a9a0 msgr2=0x563be692e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:54:15.775 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.511+0000 7fa9bec0d780 1 --2- 172.21.15.19:0/2639474284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563be692a9a0 0x563be692e970 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8051000 tx=0x7fa9a8002fb0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.775 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.512+0000 7fa9bec0d780 1 -- 172.21.15.19:0/2639474284 shutdown_connections 2023-10-06T13:54:15.775 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.512+0000 7fa9bec0d780 1 --2- 172.21.15.19:0/2639474284 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x563be6931c50 0x563be69b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.775 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.512+0000 7fa9bec0d780 1 --2- 172.21.15.19:0/2639474284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563be69302a0 0x563be68ea900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.775 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.512+0000 7fa9bec0d780 1 --2- 172.21.15.19:0/2639474284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563be692a9a0 0x563be692e970 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:15.775 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.512+0000 7fa9bec0d780 1 -- 172.21.15.19:0/2639474284 >> 172.21.15.19:0/2639474284 conn(0x563be6928310 msgr2=0x563be69291f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:54:15.775 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.512+0000 7fa9bec0d780 1 -- 172.21.15.19:0/2639474284 shutdown_connections 2023-10-06T13:54:15.776 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.512+0000 7fa9bec0d780 1 -- 172.21.15.19:0/2639474284 wait complete. 2023-10-06T13:54:15.776 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.512+0000 7fa9bec0d780 1 librados: shutdown 2023-10-06T13:54:15.776 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 1 2023-10-06T13:54:15.776 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2023-10-06T13:54:15.776 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.524+0000 7fa9bec0d780 -1 failed for service _ceph-mon._tcp 2023-10-06T13:54:15.776 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:14.524+0000 7fa9bec0d780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2023-10-06T13:54:15.776 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (237 ms) 2023-10-06T13:54:15.776 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [----------] 1 test from LibRadosMiscConnectFailure (237 ms total) 2023-10-06T13:54:15.776 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:54:15.943 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [----------] 1 test from036640 1 -- 172.21.15.19:0/880871861 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f55b801c400 con 0x55d5929d30c0 2023-10-06T13:54:15.944 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.444+0000 7f55c5036640 1 --2- 172.21.15.19:0/880871861 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f55b406a400 0x7f55b406c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:54:15.944 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.445+0000 7f55c703a640 1 --2- 172.21.15.19:0/880871861 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f55b406a400 0x7f55b406c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:54:15.944 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.445+0000 7f55c703a640 10 monclient: get_auth_request con 0x7f55b406a400 auth_method 0 2023-10-06T13:54:15.944 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.445+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f55b8058af0 con 0x55d5929d30c0 2023-10-06T13:54:15.944 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.445+0000 7f55c5036640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_map(23..23 src has 1..23) v4 2023-10-06T13:54:15.944 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.445+0000 7f55c5036640 3 client.4453.objecter handle_osd_map got epochs [23,23] > 0 2023-10-06T13:54:15.944 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.445+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding full epoch 23 2023-10-06T13:54:15.944 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.445+0000 7f55c5036640 20 client.4453.objecter dump_active .. 0 homeless 2023-10-06T13:54:15.944 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.445+0000 7f55c9d3c780 10 librados: wait_for_osdmap done waiting 2023-10-06T13:54:15.944 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.445+0000 7f55c9d3c780 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:54:15.944 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.445+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f55bc04bda0 con 0x55d5929d30c0 2023-10-06T13:54:15.945 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.446+0000 7f55c703a640 1 --2- 172.21.15.19:0/880871861 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f55b406a400 0x7f55b406c8b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55bc055000 tx=0x7f55bc04d000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:54:15.945 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.446+0000 7f55c5036640 10 client.4453.objecter ms_handle_connect 0x7f55b406a400 2023-10-06T13:54:15.945 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.499+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55b8054e30 con 0x55d5929d30c0 2023-10-06T13:54:15.945 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.499+0000 7f55c5036640 10 monclient: handle_get_version_reply finishing 1 version 23 2023-10-06T13:54:15.945 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.499+0000 7f55c5837640 10 client.4453.objecter _get_latest_version latest 23, have it 2023-10-06T13:54:15.945 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.499+0000 7f55c9d3c780 10 client.4453.objecter create_pool name=ceph_test_rados_api_asio 2023-10-06T13:54:15.945 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.499+0000 7f55c9d3c780 10 client.4453.objecter pool_op_submit 1 2023-10-06T13:54:15.945 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.499+0000 7f55c9d3c780 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:54:15.945 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:14.499+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) v4 -- 0x55d592996780 con 0x55d5929d30c0 2023-10-06T13:54:15.945 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.941+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 <== mon.1 v2:172.21.15.83:3300/0 7 ==== pool_op_reply(tid 1 (0) Success v24) v1 ==== 43+0+0 (secure 0 0 0) 0x7f55b8020020 con 0x55d5929d30c0 2023-10-06T13:54:15.974 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.941+0000 7f55c5036640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 pool_op_reply(tid 1 (0) Success v24) v1 2023-10-06T13:54:15.975 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.941+0000 7f55c5036640 10 client.4453.objecter handle_pool_op_reply pool_op_reply(tid 1 (0) Success v24) v1 2023-10-06T13:54:15.975 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.941+0000 7f55c5036640 10 client.4453.objecter have request 1 at 0x55d592aa1e10 Op: create 2023-10-06T13:54:15.975 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.941+0000 7f55c5036640 20 client.4453.objecter waiting for client to reach epoch 24 before calling back 2023-10-06T13:54:15.975 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.941+0000 7f55c5036640 10 client.4453.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:54:15.975 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.941+0000 7f55c5036640 10 monclient: _renew_subs 2023-10-06T13:54:15.975 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.941+0000 7f55c5036640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:54:15.975 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.941+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=24}) v3 -- 0x7f55b4070070 con 0x55d5929d30c0 2023-10-06T13:54:15.975 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.941+0000 7f55c5036640 10 client.4453.objecter done 2023-10-06T13:54:15.975 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.972+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 <== mon.1 v2:172.21.15.83:3300/0 8 ==== osd_map(24..24 src has 1..24) v4 ==== 2484+0+0 (secure 0 0 0) 0x7f55b805c870 con 0x55d5929d30c0 2023-10-06T13:54:15.977 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.972+0000 7f55c5036640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_map(24..24 src has 1..24) v4 2023-10-06T13:54:15.977 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.972+0000 7f55c5036640 3 client.4453.objecter handle_osd_map got epochs [24,24] > 23 2023-10-06T13:54:15.977 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.972+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 24 2023-10-06T13:54:15.977 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.972+0000 7f55c5036640 20 client.4453.objecter dump_active .. 0 homeless 2023-10-06T13:54:15.977 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.972+0000 7f55c9d3c780 10 librados: set snap read head -> 1 2023-10-06T13:54:15.977 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.972+0000 7f55c9d3c780 10 librados: writefull oid=exist nspace= 2023-10-06T13:54:15.978 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.972+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592b43b40 2023-10-06T13:54:15.978 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.972+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 24 base exist @8 precalc_pgid 0 pgid 0.0 is_write 2023-10-06T13:54:15.978 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.972+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:15.978 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.972+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.d3a59201 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:15.978 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.972+0000 7f55c9d3c780 1 --2- 172.21.15.19:0/880871861 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x55d592aa8880 0x55d592b501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T13:54:15.978 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.972+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:15.978 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.972+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid exist '@8' '@8' [writefull 0~5 in=5b] tid 2 osd.6 2023-10-06T13:54:15.978 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.973+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:15.978 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.973+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 2 2023-10-06T13:54:15.978 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.973+0000 7f55c9d3c780 15 client.4453.objecter _send_op 2 to 8.1 on osd.6 2023-10-06T13:54:23.512 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:02:26.635 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:23.576 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:02:26.699 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:23.591 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:02:26.688 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:23.597 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:02:26.694 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:23.698 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:02:26.795 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:23.867 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:02:26.990 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:23.875 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:02:26.998 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:23.933 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:02:27.056 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:23.973 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:02:27.096 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:24.187 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.973+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:2 8.1 8:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24) v8 -- 0x55d592aa6550 con 0x55d592aa8880 2023-10-06T13:54:24.187 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.973+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:24.187 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.973+0000 7f55c9d3c780 5 client.4453.objecter 1 in flight 2023-10-06T13:54:24.187 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.973+0000 7f55c803c640 1 --2- 172.21.15.19:0/880871861 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x55d592aa8880 0x55d592b501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T13:54:24.187 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.974+0000 7f55c803c640 10 monclient: get_auth_request con 0x55d592aa8880 auth_method 0 2023-10-06T13:54:24.187 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.977+0000 7f55c5036640 10 client.4453.objecter ms_handle_connect 0x55d592aa8880 2023-10-06T13:54:24.187 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:15.977+0000 7f55c803c640 1 --2- 172.21.15.19:0/880871861 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x55d592aa8880 0x55d592b501b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T13:54:24.187 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:17.315+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:24.187 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:17.315+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:24.188 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:17.315+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:24.188 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:17.315+0000 7f55affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:54:24.188 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:18.315+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:24.188 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:18.315+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:24.188 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:18.315+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:24.188 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:19.315+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:24.188 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:19.315+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:24.188 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:19.315+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:24.188 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:19.396+0000 7f55c9a9f640 10 client.4453.objecter tick 2023-10-06T13:54:24.188 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:20.316+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:24.188 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:20.316+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:24.188 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:20.316+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:24.189 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:21.316+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:24.189 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:21.316+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:24.189 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:21.316+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:24.189 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:22.316+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:24.189 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:22.316+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:24.189 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:22.316+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:24.189 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:23.316+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:24.189 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:23.316+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:24.189 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:23.316+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:24.189 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.184+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 1 ==== osd_map(25..26 src has 1..26) v4 ==== 16213+0+0 (crc 0 0 0) 0x7f55c0006c60 con 0x55d592aa8880 2023-10-06T13:54:24.189 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.184+0000 7f55c5036640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_map(25..26 src has 1..26) v4 2023-10-06T13:54:24.195 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.184+0000 7f55c5036640 3 client.4453.objecter handle_osd_map got epochs [25,26] > 24 2023-10-06T13:54:24.195 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.184+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 25 2023-10-06T13:54:24.195 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.185+0000 7f55c5036640 10 client.4453.objecter checking op 2 2023-10-06T13:54:24.195 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.185+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 25 base exist @8 precalc_pgid 0 pgid 0.0 is_write 2023-10-06T13:54:24.195 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.185+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:24.195 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.185+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 26 2023-10-06T13:54:24.195 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.185+0000 7f55c5036640 10 client.4453.objecter checking op 2 2023-10-06T13:54:24.196 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.185+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 26 base exist @8 precalc_pgid 0 pgid 0.0 is_write 2023-10-06T13:54:24.196 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.185+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:24.196 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.185+0000 7f55c5036640 20 client.4453.objecter dump_active .. 0 homeless 2023-10-06T13:54:24.196 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.185+0000 7f55c5036640 20 client.4453.objecter 2 8.d3a59201 osd.6 exist [writefull 0~5 in=5b] 2023-10-06T13:54:24.196 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2023-10-06T13:54:24.196 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 2 ==== osd_op_reply(2 exist [writefull 0~5] v26'1 uv1 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:24.196 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(2 exist [writefull 0~5] v26'1 uv1 ondisk = 0) v8 2023-10-06T13:54:24.196 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:24.196 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 2 ondisk uv 1 in 8.1 attempt 0 2023-10-06T13:54:24.196 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 0 2023-10-06T13:54:24.196 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 2 2023-10-06T13:54:24.196 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c803c640 15 client.4453.objecter _finish_op 2 2023-10-06T13:54:24.197 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:24.197 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 2 2023-10-06T13:54:24.197 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c803c640 5 client.4453.objecter 0 in flight 2023-10-06T13:54:24.197 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 10 librados: Objecter returned from writefull r=0 2023-10-06T13:54:24.197 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592b43b40 2023-10-06T13:54:24.197 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 26 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:24.197 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:24.197 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.d3a59201 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:24.197 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:24.197 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid exist '@8' '@8' [read 0~256] tid 3 osd.6 2023-10-06T13:54:24.197 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:28.319 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 3 2023-10-06T13:54:28.319 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 15 client.4453.objecter _send_op 3 to 8.1 on osd.6 2023-10-06T13:54:28.319 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:3 8.1 8:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26) v8 -- 0x55d592aa83a0 con 0x55d592aa8880 2023-10-06T13:54:28.319 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:28.319 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 5 client.4453.objecter 1 in flight 2023-10-06T13:54:28.319 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592b46920 2023-10-06T13:54:28.319 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 26 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:28.319 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:28.319 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.a7bbaa1 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:28.319 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:28.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid noexist '@8' '@8' [read 0~256] tid 4 osd.6 2023-10-06T13:54:28.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:28.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 4 2023-10-06T13:54:28.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 15 client.4453.objecter _send_op 4 to 8.1 on osd.6 2023-10-06T13:54:28.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:4 8.1 8:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26) v8 -- 0x55d592b44030 con 0x55d592aa8880 2023-10-06T13:54:28.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 5 2023-10-06T13:54:28.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.193+0000 7f55c9d3c780 5 client.4453.objecter 2 in flight 2023-10-06T13:54:28.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.316+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:28.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.316+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:28.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.316+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:28.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:24.397+0000 7f55c9a9f640 10 client.4453.objecter tick 2023-10-06T13:54:28.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:25.316+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:28.321 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:25.316+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:28.321 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:25.316+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:28.321 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:26.316+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:28.321 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:26.316+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:28.321 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:26.316+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:28.321 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:27.316+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:28.321 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:27.316+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:28.321 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:27.316+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:28.321 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:27.316+0000 7f55affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:54:32.409 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:28.316+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:32.409 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:28.316+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:32.410 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:28.316+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:32.410 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:29.316+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:32.410 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:29.316+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:32.410 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:29.316+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:32.410 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:29.398+0000 7f55c9a9f640 10 client.4453.objecter tick 2023-10-06T13:54:32.410 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:30.317+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:32.410 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:30.317+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:32.410 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:30.317+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:32.410 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:31.317+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:32.411 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:31.317+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:32.411 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:31.317+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:32.411 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.317+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:32.411 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.317+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:32.411 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.317+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:32.411 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 3 ==== osd_map(27..32 src has 1..32) v4 ==== 5655+0+0 (crc 0 0 0) 0x7f55c0011930 con 0x55d592aa8880 2023-10-06T13:54:32.411 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_map(27..32 src has 1..32) v4 2023-10-06T13:54:32.411 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 3 client.4453.objecter handle_osd_map got epochs [27,32] > 26 2023-10-06T13:54:32.411 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 27 2023-10-06T13:54:32.411 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 10 client.4453.objecter checking op 3 2023-10-06T13:54:32.411 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 27 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:32.411 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:32.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 10 client.4453.objecter checking op 4 2023-10-06T13:54:32.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 27 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:32.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:32.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 28 2023-10-06T13:54:32.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 10 client.4453.objecter checking op 3 2023-10-06T13:54:32.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 28 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:32.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:32.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 10 client.4453.objecter checking op 4 2023-10-06T13:54:32.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 28 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:32.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:32.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 29 2023-10-06T13:54:32.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 10 client.4453.objecter checking op 3 2023-10-06T13:54:32.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 29 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:32.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:32.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 10 client.4453.objecter checking op 4 2023-10-06T13:54:32.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 29 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:32.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:32.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 30 2023-10-06T13:54:32.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 10 client.4453.objecter checking op 3 2023-10-06T13:54:32.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 30 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:32.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:32.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 10 client.4453.objecter checking op 4 2023-10-06T13:54:32.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 30 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:32.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:32.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 31 2023-10-06T13:54:32.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 10 client.4453.objecter checking op 3 2023-10-06T13:54:32.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 31 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:32.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:32.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 10 client.4453.objecter checking op 4 2023-10-06T13:54:32.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 31 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:32.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:32.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 32 2023-10-06T13:54:32.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 10 client.4453.objecter checking op 3 2023-10-06T13:54:32.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 32 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:32.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:32.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 10 client.4453.objecter checking op 4 2023-10-06T13:54:32.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 32 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:32.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:32.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter dump_active .. 0 homeless 2023-10-06T13:54:32.431 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter 3 8.d3a59201 osd.6 exist [read 0~256] 2023-10-06T13:54:32.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.407+0000 7f55c5036640 20 client.4453.objecter 4 8.a7bbaa1 osd.6 noexist [read 0~256] 2023-10-06T13:54:32.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 4 ==== osd_op_reply(3 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:32.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(3 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2023-10-06T13:54:32.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:32.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 3 ondisk uv 1 in 8.1 attempt 0 2023-10-06T13:54:32.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 5 2023-10-06T13:54:32.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 3 2023-10-06T13:54:32.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 15 client.4453.objecter _finish_op 3 2023-10-06T13:54:32.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 5 2023-10-06T13:54:32.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 3 2023-10-06T13:54:32.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 5 client.4453.objecter 1 in flight 2023-10-06T13:54:32.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 5 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:32.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-10-06T13:54:32.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:32.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 4 ondisk uv 0 in 8.1 attempt 0 2023-10-06T13:54:32.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 10 client.4453.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-10-06T13:54:32.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 0 2023-10-06T13:54:32.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 4 2023-10-06T13:54:32.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 15 client.4453.objecter _finish_op 4 2023-10-06T13:54:32.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:32.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 4 2023-10-06T13:54:32.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c803c640 5 client.4453.objecter 0 in flight 2023-10-06T13:54:32.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (8236 ms) 2023-10-06T13:54:32.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2023-10-06T13:54:32.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592b43b40 2023-10-06T13:54:32.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 32 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:32.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:32.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.d3a59201 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:32.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:35.674 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid exist '@8' '@8' [read 0~256] tid 5 osd.6 2023-10-06T13:54:35.674 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:35.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 5 2023-10-06T13:54:35.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 15 client.4453.objecter _send_op 5 to 8.1 on osd.6 2023-10-06T13:54:35.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:5 8.1 8:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x55d592b46920 con 0x55d592aa8880 2023-10-06T13:54:35.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:35.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 5 client.4453.objecter 1 in flight 2023-10-06T13:54:35.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592b46d80 2023-10-06T13:54:35.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 32 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:35.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:35.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.a7bbaa1 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:35.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:35.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid noexist '@8' '@8' [read 0~256] tid 6 osd.6 2023-10-06T13:54:35.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:35.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 6 2023-10-06T13:54:35.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 15 client.4453.objecter _send_op 6 to 8.1 on osd.6 2023-10-06T13:54:35.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:6 8.1 8:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) v8 -- 0x55d592b47270 con 0x55d592aa8880 2023-10-06T13:54:35.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 5 2023-10-06T13:54:35.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:32.429+0000 7f55c9d3c780 5 client.4453.objecter 2 in flight 2023-10-06T13:54:35.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:33.317+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:35.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:33.317+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:35.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:33.317+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:35.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:34.317+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:35.679 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:34.317+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:35.679 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:34.317+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:35.679 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:34.399+0000 7f55c9a9f640 10 client.4453.objecter tick 2023-10-06T13:54:35.679 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.317+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:35.679 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.317+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:35.679 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.317+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:35.680 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 6 ==== osd_map(33..37 src has 1..37) v4 ==== 2427+0+0 (crc 0 0 0) 0x7f55c0015e00 con 0x55d592aa8880 2023-10-06T13:54:35.680 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 7 ==== osd_op_reply(5 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:35.680 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_map(33..37 src has 1..37) v4 2023-10-06T13:54:35.680 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(5 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2023-10-06T13:54:35.680 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:35.680 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 3 client.4453.objecter handle_osd_map got epochs [33,37] > 32 2023-10-06T13:54:35.680 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 33 2023-10-06T13:54:35.681 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 10 client.4453.objecter checking op 5 2023-10-06T13:54:35.681 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 33 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:35.681 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:35.681 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 10 client.4453.objecter checking op 6 2023-10-06T13:54:35.681 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 33 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:35.681 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:35.682 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 34 2023-10-06T13:54:35.682 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 10 client.4453.objecter checking op 5 2023-10-06T13:54:35.682 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 34 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:35.682 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:35.682 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 10 client.4453.objecter checking op 6 2023-10-06T13:54:35.682 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 34 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:35.683 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:35.683 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 35 2023-10-06T13:54:35.683 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 10 client.4453.objecter checking op 5 2023-10-06T13:54:35.683 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 35 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:35.683 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:35.683 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 10 client.4453.objecter checking op 6 2023-10-06T13:54:35.684 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 35 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:35.684 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:35.684 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 36 2023-10-06T13:54:35.684 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 10 client.4453.objecter checking op 5 2023-10-06T13:54:35.684 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 36 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:35.684 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:35.685 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 10 client.4453.objecter checking op 6 2023-10-06T13:54:35.685 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 36 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:35.685 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:35.685 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 37 2023-10-06T13:54:35.685 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 10 client.4453.objecter checking op 5 2023-10-06T13:54:35.686 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 37 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:35.686 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:35.686 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 10 client.4453.objecter checking op 6 2023-10-06T13:54:35.686 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 37 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:35.686 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:35.686 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter dump_active .. 0 homeless 2023-10-06T13:54:35.687 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter 5 8.d3a59201 osd.6 exist [read 0~256] 2023-10-06T13:54:35.687 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c5036640 20 client.4453.objecter 6 8.a7bbaa1 osd.6 noexist [read 0~256] 2023-10-06T13:54:35.687 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 5 ondisk uv 1 in 8.1 attempt 0 2023-10-06T13:54:35.687 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 5 2023-10-06T13:54:35.687 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 5 2023-10-06T13:54:35.687 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c803c640 15 client.4453.objecter _finish_op 5 2023-10-06T13:54:35.688 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 5 2023-10-06T13:54:35.688 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 5 2023-10-06T13:54:35.688 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.671+0000 7f55c803c640 5 client.4453.objecter 1 in flight 2023-10-06T13:54:35.688 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 8 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:35.688 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-10-06T13:54:35.688 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:35.689 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 6 ondisk uv 0 in 8.1 attempt 0 2023-10-06T13:54:35.689 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c803c640 10 client.4453.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-10-06T13:54:35.689 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 0 2023-10-06T13:54:35.689 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 6 2023-10-06T13:54:35.689 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c803c640 15 client.4453.objecter _finish_op 6 2023-10-06T13:54:35.689 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:35.690 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 6 2023-10-06T13:54:35.690 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c803c640 5 client.4453.objecter 0 in flight 2023-10-06T13:54:35.690 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (3243 ms) 2023-10-06T13:54:35.690 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2023-10-06T13:54:35.690 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592b46d80 2023-10-06T13:54:35.690 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 37 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:35.691 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:35.691 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.d3a59201 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:35.691 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:35.691 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid exist '@8' '@8' [read 0~256] tid 7 osd.6 2023-10-06T13:54:35.691 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:35.691 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 7 2023-10-06T13:54:35.691 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 15 client.4453.objecter _send_op 7 to 8.1 on osd.6 2023-10-06T13:54:35.692 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:7 8.1 8:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) v8 -- 0x55d592b43b40 con 0x55d592aa8880 2023-10-06T13:54:35.692 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:35.692 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 5 client.4453.objecter 1 in flight 2023-10-06T13:54:35.692 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592a804f0 2023-10-06T13:54:35.692 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 37 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:35.693 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:35.693 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.a7bbaa1 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:35.693 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:35.693 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid noexist '@8' '@8' [read 0~256] tid 8 osd.6 2023-10-06T13:54:35.693 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:35.693 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 8 2023-10-06T13:54:35.694 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 15 client.4453.objecter _send_op 8 to 8.1 on osd.6 2023-10-06T13:54:35.694 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:8 8.1 8:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) v8 -- 0x55d592a809e0 con 0x55d592aa8880 2023-10-06T13:54:35.694 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 5 2023-10-06T13:54:35.741 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.672+0000 7f55c9d3c780 5 client.4453.objecter 2 in flight 2023-10-06T13:54:35.741 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (67 ms) 2023-10-06T13:54:35.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2023-10-06T13:54:35.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 9 ==== osd_op_reply(7 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:35.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(7 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2023-10-06T13:54:35.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:35.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 7 ondisk uv 1 in 8.1 attempt 0 2023-10-06T13:54:35.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 5 2023-10-06T13:54:35.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 7 2023-10-06T13:54:35.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 15 client.4453.objecter _finish_op 7 2023-10-06T13:54:35.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 5 2023-10-06T13:54:35.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 7 2023-10-06T13:54:35.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 5 client.4453.objecter 1 in flight 2023-10-06T13:54:35.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 10 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:35.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-10-06T13:54:35.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:35.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 8 ondisk uv 0 in 8.1 attempt 0 2023-10-06T13:54:35.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 10 client.4453.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-10-06T13:54:35.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 0 2023-10-06T13:54:35.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 8 2023-10-06T13:54:35.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 15 client.4453.objecter _finish_op 8 2023-10-06T13:54:35.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:35.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 8 2023-10-06T13:54:35.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c803c640 5 client.4453.objecter 0 in flight 2023-10-06T13:54:35.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2023-10-06T13:54:35.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 20 librados: queue_aio_write 0x55d592985dc0 completion 0x55d592b477f0 write_seq 1 2023-10-06T13:54:35.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592b46d80 2023-10-06T13:54:35.746 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 37 base exist @8 precalc_pgid 0 pgid 0.0 is_write 2023-10-06T13:54:35.746 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:35.746 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.d3a59201 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:36.884 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:36.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid exist '@8' '@8' [write 0~5 in=5b] tid 9 osd.6 2023-10-06T13:54:36.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:36.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 9 2023-10-06T13:54:36.987 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 15 client.4453.objecter _send_op 9 to 8.1 on osd.6 2023-10-06T13:54:36.987 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:9 8.1 8:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37) v8 -- 0x55d592a7fe80 con 0x55d592aa8880 2023-10-06T13:54:36.987 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:36.987 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 5 client.4453.objecter 1 in flight 2023-10-06T13:54:36.987 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:35.739+0000 7f55c9d3c780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2023-10-06T13:54:36.987 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.317+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:37.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.317+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:37.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.317+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:37.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 11 ==== osd_op_reply(9 exist [write 0~5] v37'2 uv2 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:37.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(9 exist [write 0~5] v37'2 uv2 ondisk = 0) v8 2023-10-06T13:54:37.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:37.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 9 ondisk uv 2 in 8.1 attempt 0 2023-10-06T13:54:37.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 0 2023-10-06T13:54:37.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 9 2023-10-06T13:54:37.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c803c640 15 client.4453.objecter _finish_op 9 2023-10-06T13:54:37.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:37.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 9 2023-10-06T13:54:37.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c803c640 5 client.4453.objecter 0 in flight 2023-10-06T13:54:37.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c803c640 20 librados: complete_aio_write 0x55d592b477f0 2023-10-06T13:54:37.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (1143 ms) 2023-10-06T13:54:37.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2023-10-06T13:54:37.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c9d3c780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2023-10-06T13:54:37.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c9d3c780 20 librados: queue_aio_write 0x55d592985dc0 completion 0x55d592a80230 write_seq 2 2023-10-06T13:54:37.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592b46d80 2023-10-06T13:54:37.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 37 base exist @8 precalc_pgid 0 pgid 0.0 is_write 2023-10-06T13:54:37.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:38.858 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.d3a59201 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:38.858 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:38.858 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid exist '@8' '@8' [write 0~5 in=5b] tid 10 osd.6 2023-10-06T13:54:38.858 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:38.858 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 10 2023-10-06T13:54:38.858 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c9d3c780 15 client.4453.objecter _send_op 10 to 8.1 on osd.6 2023-10-06T13:54:38.858 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.882+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:10 8.1 8:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37) v8 -- 0x55d592a7fe80 con 0x55d592aa8880 2023-10-06T13:54:38.858 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.883+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:38.859 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.883+0000 7f55c9d3c780 5 client.4453.objecter 1 in flight 2023-10-06T13:54:38.859 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:36.883+0000 7f55c9d3c780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2023-10-06T13:54:38.859 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:37.317+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:38.859 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:37.317+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:38.859 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:37.317+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:38.859 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:37.317+0000 7f55affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T13:54:38.859 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.317+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:38.859 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.317+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:38.859 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.317+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:38.859 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 12 ==== osd_map(38..40 src has 1..40) v4 ==== 1978+0+0 (crc 0 0 0) 0x7f55c0011d30 con 0x55d592aa8880 2023-10-06T13:54:38.859 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_map(38..40 src has 1..40) v4 2023-10-06T13:54:38.859 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 3 client.4453.objecter handle_osd_map got epochs [38,40] > 37 2023-10-06T13:54:38.860 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 38 2023-10-06T13:54:38.860 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 10 client.4453.objecter checking op 10 2023-10-06T13:54:38.860 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 38 base exist @8 precalc_pgid 0 pgid 0.0 is_write 2023-10-06T13:54:38.860 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:38.860 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 39 2023-10-06T13:54:38.860 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 10 client.4453.objecter checking op 10 2023-10-06T13:54:38.860 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 39 base exist @8 precalc_pgid 0 pgid 0.0 is_write 2023-10-06T13:54:38.860 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:38.860 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 40 2023-10-06T13:54:38.860 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 10 client.4453.objecter checking op 10 2023-10-06T13:54:41.256 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 40 base exist @8 precalc_pgid 0 pgid 0.0 is_write 2023-10-06T13:54:41.256 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:41.256 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 20 client.4453.objecter dump_active .. 0 homeless 2023-10-06T13:54:41.256 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:38.856+0000 7f55c5036640 20 client.4453.objecter 10 8.d3a59201 osd.6 exist [write 0~5 in=5b] 2023-10-06T13:54:41.256 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:39.317+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:41.256 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:39.317+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:41.257 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:39.317+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:41.257 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:39.400+0000 7f55c9a9f640 10 client.4453.objecter tick 2023-10-06T13:54:41.257 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:40.317+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:41.257 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:40.317+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:41.257 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:40.317+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:41.257 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 13 ==== osd_op_reply(10 exist [write 0~5] v40'3 uv3 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:41.257 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(10 exist [write 0~5] v40'3 uv3 ondisk = 0) v8 2023-10-06T13:54:41.257 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:41.257 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 10 ondisk uv 3 in 8.1 attempt 0 2023-10-06T13:54:41.257 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 0 2023-10-06T13:54:41.257 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 10 2023-10-06T13:54:41.258 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c803c640 15 client.4453.objecter _finish_op 10 2023-10-06T13:54:41.258 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:41.258 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 10 2023-10-06T13:54:41.258 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c803c640 5 client.4453.objecter 0 in flight 2023-10-06T13:54:41.258 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c803c640 20 librados: complete_aio_write 0x55d592a80230 2023-10-06T13:54:41.258 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (4372 ms) 2023-10-06T13:54:41.258 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2023-10-06T13:54:41.258 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2023-10-06T13:54:41.258 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 20 librados: queue_aio_write 0x55d592985dc0 completion 0x55d592a80870 write_seq 3 2023-10-06T13:54:41.258 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592b46d80 2023-10-06T13:54:41.258 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 40 base exist @8 precalc_pgid 0 pgid 0.0 is_write 2023-10-06T13:54:41.259 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:41.259 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.d3a59201 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:41.259 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:41.259 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid exist '@8' '@8' [write 0~5 in=5b] tid 11 osd.6 2023-10-06T13:54:41.365 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:41.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 11 2023-10-06T13:54:41.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 15 client.4453.objecter _send_op 11 to 8.1 on osd.6 2023-10-06T13:54:41.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:11 8.1 8:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55d592a83ff0 con 0x55d592aa8880 2023-10-06T13:54:41.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:41.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 5 client.4453.objecter 1 in flight 2023-10-06T13:54:41.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.254+0000 7f55c9d3c780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2023-10-06T13:54:41.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 14 ==== osd_map(41..42 src has 1..42) v4 ==== 2684+0+0 (crc 0 0 0) 0x7f55c0011300 con 0x55d592aa8880 2023-10-06T13:54:41.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_map(41..42 src has 1..42) v4 2023-10-06T13:54:41.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 3 client.4453.objecter handle_osd_map got epochs [41,42] > 40 2023-10-06T13:54:41.366 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 41 2023-10-06T13:54:41.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 10 client.4453.objecter checking op 11 2023-10-06T13:54:41.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 41 base exist @8 precalc_pgid 0 pgid 0.0 is_write 2023-10-06T13:54:41.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:41.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 42 2023-10-06T13:54:41.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 10 client.4453.objecter checking op 11 2023-10-06T13:54:41.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 42 base exist @8 precalc_pgid 0 pgid 0.0 is_write 2023-10-06T13:54:41.367 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:41.368 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 10 client.4453.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:54:41.368 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 10 monclient: _renew_subs 2023-10-06T13:54:41.368 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:54:41.368 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=43}) v3 -- 0x7f55b4079660 con 0x55d5929d30c0 2023-10-06T13:54:41.368 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 20 client.4453.objecter dump_active .. 0 homeless 2023-10-06T13:54:41.368 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.278+0000 7f55c5036640 20 client.4453.objecter 11 8.d3a59201 osd.6 exist [write 0~5 in=5b] 2023-10-06T13:54:41.368 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.318+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:41.368 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.318+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:41.369 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.318+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:41.369 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.360+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 15 ==== osd_op_reply(11 exist [write 0~5] v42'4 uv4 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:41.369 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.360+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(11 exist [write 0~5] v42'4 uv4 ondisk = 0) v8 2023-10-06T13:54:41.369 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.360+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:41.369 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.360+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 11 ondisk uv 4 in 8.1 attempt 0 2023-10-06T13:54:41.369 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.360+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 0 2023-10-06T13:54:41.369 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.360+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 11 2023-10-06T13:54:41.369 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.360+0000 7f55c803c640 15 client.4453.objecter _finish_op 11 2023-10-06T13:54:41.369 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.360+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:41.369 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.360+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 11 2023-10-06T13:54:41.369 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.360+0000 7f55c803c640 5 client.4453.objecter 0 in flight 2023-10-06T13:54:41.369 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.360+0000 7f55c803c640 20 librados: complete_aio_write 0x55d592a80870 2023-10-06T13:54:41.370 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (107 ms) 2023-10-06T13:54:41.370 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2023-10-06T13:54:41.370 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592a83980 2023-10-06T13:54:41.370 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 42 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:41.370 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:41.370 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.d3a59201 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:41.370 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:41.370 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid exist '@8' '@8' [read 0~0] tid 12 osd.6 2023-10-06T13:54:41.370 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:41.370 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 12 2023-10-06T13:54:41.370 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 15 client.4453.objecter _send_op 12 to 8.1 on osd.6 2023-10-06T13:54:41.371 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:12 8.1 8:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55d592a83e70 con 0x55d592aa8880 2023-10-06T13:54:41.371 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:41.371 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 5 client.4453.objecter 1 in flight 2023-10-06T13:54:41.371 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592a80e10 2023-10-06T13:54:41.371 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 42 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:41.371 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:41.371 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.a7bbaa1 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:41.371 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:41.406 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid noexist '@8' '@8' [read 0~0] tid 13 osd.6 2023-10-06T13:54:41.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:41.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 13 2023-10-06T13:54:41.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 15 client.4453.objecter _send_op 13 to 8.1 on osd.6 2023-10-06T13:54:41.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:13 8.1 8:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55d592a81300 con 0x55d592aa8880 2023-10-06T13:54:41.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 5 2023-10-06T13:54:41.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.361+0000 7f55c9d3c780 5 client.4453.objecter 2 in flight 2023-10-06T13:54:41.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 16 ==== osd_op_reply(12 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:41.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(12 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2023-10-06T13:54:41.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:41.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 12 ondisk uv 4 in 8.1 attempt 0 2023-10-06T13:54:41.407 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 5 2023-10-06T13:54:41.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 12 2023-10-06T13:54:41.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 15 client.4453.objecter _finish_op 12 2023-10-06T13:54:41.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 5 2023-10-06T13:54:41.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 12 2023-10-06T13:54:41.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 5 client.4453.objecter 1 in flight 2023-10-06T13:54:41.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 17 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:41.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-10-06T13:54:41.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:41.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 13 ondisk uv 0 in 8.1 attempt 0 2023-10-06T13:54:41.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 10 client.4453.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-10-06T13:54:41.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 0 2023-10-06T13:54:41.408 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 13 2023-10-06T13:54:41.409 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 15 client.4453.objecter _finish_op 13 2023-10-06T13:54:41.409 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:41.409 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 13 2023-10-06T13:54:41.409 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c803c640 5 client.4453.objecter 0 in flight 2023-10-06T13:54:41.409 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (43 ms) 2023-10-06T13:54:41.468 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2023-10-06T13:54:41.469 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592a83980 2023-10-06T13:54:41.469 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 42 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:41.469 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:41.469 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.d3a59201 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:41.469 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:41.469 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid exist '@8' '@8' [read 0~0] tid 14 osd.6 2023-10-06T13:54:41.469 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:41.469 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 14 2023-10-06T13:54:41.469 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 15 client.4453.objecter _send_op 14 to 8.1 on osd.6 2023-10-06T13:54:41.469 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:14 8.1 8:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55d592a83e70 con 0x55d592aa8880 2023-10-06T13:54:41.470 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:41.470 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 5 client.4453.objecter 1 in flight 2023-10-06T13:54:41.470 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592a80e10 2023-10-06T13:54:41.470 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 42 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:41.470 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:41.470 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.a7bbaa1 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:41.470 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:41.470 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid noexist '@8' '@8' [read 0~0] tid 15 osd.6 2023-10-06T13:54:41.470 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:41.471 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 15 2023-10-06T13:54:41.471 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 15 client.4453.objecter _send_op 15 to 8.1 on osd.6 2023-10-06T13:54:41.471 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:15 8.1 8:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55d592a81300 con 0x55d592aa8880 2023-10-06T13:54:41.471 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 5 2023-10-06T13:54:41.471 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.404+0000 7f55c9d3c780 5 client.4453.objecter 2 in flight 2023-10-06T13:54:41.471 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 <== mon.1 v2:172.21.15.83:3300/0 9 ==== osd_map(43..43 src has 1..43) v4 ==== 691+0+0 (secure 0 0 0) 0x7f55b8055200 con 0x55d5929d30c0 2023-10-06T13:54:41.471 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_map(43..43 src has 1..43) v4 2023-10-06T13:54:42.251 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 3 client.4453.objecter handle_osd_map got epochs [43,43] > 42 2023-10-06T13:54:42.251 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 3 client.4453.objecter handle_osd_map decoding incremental epoch 43 2023-10-06T13:54:42.251 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 10 client.4453.objecter checking op 14 2023-10-06T13:54:42.251 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 43 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:42.251 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:42.252 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 10 client.4453.objecter checking op 15 2023-10-06T13:54:42.252 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 20 client.4453.objecter _calc_target epoch 43 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:42.252 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:42.252 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 10 client.4453.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:54:42.252 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 10 monclient: _renew_subs 2023-10-06T13:54:42.252 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T13:54:42.252 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=44}) v3 -- 0x7f55b4070c60 con 0x55d5929d30c0 2023-10-06T13:54:42.252 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 20 client.4453.objecter dump_active .. 0 homeless 2023-10-06T13:54:42.252 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 20 client.4453.objecter 14 8.d3a59201 osd.6 exist [read 0~0] 2023-10-06T13:54:42.252 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:41.467+0000 7f55c5036640 20 client.4453.objecter 15 8.a7bbaa1 osd.6 noexist [read 0~0] 2023-10-06T13:54:42.252 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c5036640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 18 ==== osd_map(43..43 src has 1..43) v4 ==== 691+0+0 (crc 0 0 0) 0x7f55c0001900 con 0x55d592aa8880 2023-10-06T13:54:42.253 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 19 ==== osd_op_reply(14 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:42.253 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c5036640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_map(43..43 src has 1..43) v4 2023-10-06T13:54:42.253 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(14 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2023-10-06T13:54:42.253 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:42.253 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c5036640 3 client.4453.objecter handle_osd_map ignoring epochs [43,43] <= 43 2023-10-06T13:54:42.253 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c5036640 10 client.4453.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T13:54:42.253 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c5036640 20 client.4453.objecter dump_active .. 0 homeless 2023-10-06T13:54:42.253 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c5036640 20 client.4453.objecter 14 8.d3a59201 osd.6 exist [read 0~0] 2023-10-06T13:54:42.253 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c5036640 20 client.4453.objecter 15 8.a7bbaa1 osd.6 noexist [read 0~0] 2023-10-06T13:54:42.253 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 14 ondisk uv 4 in 8.1 attempt 0 2023-10-06T13:54:42.253 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 5 2023-10-06T13:54:42.253 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 14 2023-10-06T13:54:42.254 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 15 client.4453.objecter _finish_op 14 2023-10-06T13:54:42.254 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 5 2023-10-06T13:54:42.254 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 14 2023-10-06T13:54:42.254 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 5 client.4453.objecter 1 in flight 2023-10-06T13:54:42.254 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 20 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:42.254 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-10-06T13:54:42.254 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:42.254 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 15 ondisk uv 0 in 8.1 attempt 0 2023-10-06T13:54:42.254 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 10 client.4453.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-10-06T13:54:42.254 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 0 2023-10-06T13:54:42.254 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 15 2023-10-06T13:54:42.254 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 15 client.4453.objecter _finish_op 15 2023-10-06T13:54:42.255 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:42.255 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 15 2023-10-06T13:54:42.255 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c803c640 5 client.4453.objecter 0 in flight 2023-10-06T13:54:42.255 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (845 ms) 2023-10-06T13:54:42.255 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2023-10-06T13:54:42.255 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592a83980 2023-10-06T13:54:42.255 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 43 base exist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:42.255 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:42.255 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.d3a59201 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:42.255 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:42.255 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid exist '@8' '@8' [read 0~0] tid 16 osd.6 2023-10-06T13:54:42.255 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:42.255 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 16 2023-10-06T13:54:42.256 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 15 client.4453.objecter _send_op 16 to 8.1 on osd.6 2023-10-06T13:54:42.256 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:16 8.1 8:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55d592a83e70 con 0x55d592aa8880 2023-10-06T13:54:42.256 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:42.256 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 5 client.4453.objecter 1 in flight 2023-10-06T13:54:42.309 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592a811b0 2023-10-06T13:54:42.309 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 43 base noexist @8 precalc_pgid 0 pgid 0.0 is_read 2023-10-06T13:54:42.309 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target noexist @8 -> pgid 8.a7bbaa1 2023-10-06T13:54:42.309 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.a7bbaa1 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:42.309 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:42.309 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid noexist '@8' '@8' [read 0~0] tid 17 osd.6 2023-10-06T13:54:42.309 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:42.309 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 17 2023-10-06T13:54:42.309 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 15 client.4453.objecter _send_op 17 to 8.1 on osd.6 2023-10-06T13:54:42.310 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:17 8.1 8:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55d592a843f0 con 0x55d592aa8880 2023-10-06T13:54:42.310 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 5 2023-10-06T13:54:42.310 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.249+0000 7f55c9d3c780 5 client.4453.objecter 2 in flight 2023-10-06T13:54:42.310 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.295+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 21 ==== osd_op_reply(16 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:42.310 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.295+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(16 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2023-10-06T13:54:42.310 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.295+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:42.310 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.295+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 16 ondisk uv 4 in 8.1 attempt 0 2023-10-06T13:54:42.310 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.295+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 5 2023-10-06T13:54:42.310 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.295+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 16 2023-10-06T13:54:42.310 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.295+0000 7f55c803c640 15 client.4453.objecter _finish_op 16 2023-10-06T13:54:42.310 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.295+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 5 2023-10-06T13:54:42.310 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.295+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 16 2023-10-06T13:54:42.311 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.295+0000 7f55c803c640 5 client.4453.objecter 1 in flight 2023-10-06T13:54:42.311 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 22 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:42.311 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-10-06T13:54:42.311 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:42.311 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 17 ondisk uv 0 in 8.1 attempt 0 2023-10-06T13:54:42.311 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c803c640 10 client.4453.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-10-06T13:54:42.352 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 0 2023-10-06T13:54:42.352 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 17 2023-10-06T13:54:42.352 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c803c640 15 client.4453.objecter _finish_op 17 2023-10-06T13:54:42.352 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:42.352 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 17 2023-10-06T13:54:42.352 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c803c640 5 client.4453.objecter 0 in flight 2023-10-06T13:54:42.352 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (58 ms) 2023-10-06T13:54:42.352 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2023-10-06T13:54:42.352 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c9d3c780 20 librados: queue_aio_write 0x55d592985dc0 completion 0x55d592a81060 write_seq 4 2023-10-06T13:54:42.353 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592a811b0 2023-10-06T13:54:42.353 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 43 base exist @8 precalc_pgid 0 pgid 0.0 is_write 2023-10-06T13:54:42.353 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:42.353 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.d3a59201 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:42.353 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:42.353 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid exist '@8' '@8' [writefull 0~5 in=5b] tid 18 osd.6 2023-10-06T13:54:42.353 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:42.353 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 18 2023-10-06T13:54:42.353 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c9d3c780 15 client.4453.objecter _send_op 18 to 8.1 on osd.6 2023-10-06T13:54:42.353 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:18 8.1 8:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55d592a843f0 con 0x55d592aa8880 2023-10-06T13:54:42.353 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:42.353 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.307+0000 7f55c9d3c780 5 client.4453.objecter 1 in flight 2023-10-06T13:54:42.354 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.318+0000 7f55affff640 10 monclient: tick 2023-10-06T13:54:42.354 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.318+0000 7f55affff640 10 monclient: _check_auth_tickets 2023-10-06T13:54:42.354 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.318+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T13:54:42.354 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 23 ==== osd_op_reply(18 exist [writefull 0~5] v43'5 uv5 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:42.354 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(18 exist [writefull 0~5] v43'5 uv5 ondisk = 0) v8 2023-10-06T13:54:42.354 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:42.354 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 18 ondisk uv 5 in 8.1 attempt 0 2023-10-06T13:54:42.354 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 0 2023-10-06T13:54:42.390 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 18 2023-10-06T13:54:42.390 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c803c640 15 client.4453.objecter _finish_op 18 2023-10-06T13:54:42.390 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:42.391 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 18 2023-10-06T13:54:42.391 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c803c640 5 client.4453.objecter 0 in flight 2023-10-06T13:54:42.391 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c803c640 20 librados: complete_aio_write 0x55d592a81060 2023-10-06T13:54:42.391 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (43 ms) 2023-10-06T13:54:42.391 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2023-10-06T13:54:42.391 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c9d3c780 20 librados: queue_aio_write 0x55d592985dc0 completion 0x55d592a81730 write_seq 5 2023-10-06T13:54:42.391 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592a81190 2023-10-06T13:54:42.391 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 43 base exist @8 precalc_pgid 0 pgid 0.0 is_write 2023-10-06T13:54:42.391 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:42.391 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.d3a59201 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:42.392 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:42.392 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid exist '@8' '@8' [writefull 0~5 in=5b] tid 19 osd.6 2023-10-06T13:54:42.392 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:42.392 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 19 2023-10-06T13:54:42.392 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c9d3c780 15 client.4453.objecter _send_op 19 to 8.1 on osd.6 2023-10-06T13:54:42.392 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:19 8.1 8:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55d592a843f0 con 0x55d592aa8880 2023-10-06T13:54:42.392 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:42.392 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.350+0000 7f55c9d3c780 5 client.4453.objecter 1 in flight 2023-10-06T13:54:42.392 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 24 ==== osd_op_reply(19 exist [writefull 0~5] v43'6 uv6 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:42.392 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(19 exist [writefull 0~5] v43'6 uv6 ondisk = 0) v8 2023-10-06T13:54:42.393 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:42.393 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 19 ondisk uv 6 in 8.1 attempt 0 2023-10-06T13:54:42.393 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 0 2023-10-06T13:54:42.393 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 19 2023-10-06T13:54:42.393 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c803c640 15 client.4453.objecter _finish_op 19 2023-10-06T13:54:42.393 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:42.393 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 19 2023-10-06T13:54:42.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c803c640 5 client.4453.objecter 0 in flight 2023-10-06T13:54:42.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c803c640 20 librados: complete_aio_write 0x55d592a81730 2023-10-06T13:54:42.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (38 ms) 2023-10-06T13:54:42.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2023-10-06T13:54:42.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c9d3c780 20 librados: queue_aio_write 0x55d592985dc0 completion 0x55d592a81840 write_seq 6 2023-10-06T13:54:42.412 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c9d3c780 10 client.4453.objecter _op_submit op 0x55d592a81190 2023-10-06T13:54:42.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c9d3c780 20 client.4453.objecter _calc_target epoch 43 base exist @8 precalc_pgid 0 pgid 0.0 is_write 2023-10-06T13:54:42.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c9d3c780 20 client.4453.objecter _calc_target target exist @8 -> pgid 8.d3a59201 2023-10-06T13:54:42.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c9d3c780 10 client.4453.objecter _calc_target raw pgid 8.d3a59201 -> actual 8.1 acting [6,4] primary 6 2023-10-06T13:54:42.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c9d3c780 20 client.4453.objecter _get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:42.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c9d3c780 10 client.4453.objecter _op_submit oid exist '@8' '@8' [writefull 0~5 in=5b] tid 20 osd.6 2023-10-06T13:54:42.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c9d3c780 20 client.4453.objecter get_session s=0x55d592927d10 osd=6 3 2023-10-06T13:54:42.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c9d3c780 15 client.4453.objecter _session_op_assign 6 20 2023-10-06T13:54:42.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c9d3c780 15 client.4453.objecter _send_op 20 to 8.1 on osd.6 2023-10-06T13:54:42.413 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- osd_op(unknown.0.0:20 8.1 8:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55d592a843f0 con 0x55d592aa8880 2023-10-06T13:54:42.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:42.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.388+0000 7f55c9d3c780 5 client.4453.objecter 1 in flight 2023-10-06T13:54:42.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 <== osd.6 v2:172.21.15.83:6802/86124 25 ==== osd_op_reply(20 exist [writefull 0~5] v43'7 uv7 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f55c0001680 con 0x55d592aa8880 2023-10-06T13:54:42.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c803c640 10 client.4453.objecter ms_dispatch 0x55d59297cb30 osd_op_reply(20 exist [writefull 0~5] v43'7 uv7 ondisk = 0) v8 2023-10-06T13:54:42.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c803c640 10 client.4453.objecter in handle_osd_op_reply 2023-10-06T13:54:42.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c803c640 7 client.4453.objecter handle_osd_op_reply 20 ondisk uv 7 in 8.1 attempt 0 2023-10-06T13:54:42.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c803c640 10 client.4453.objecter op 0 rval 0 len 0 2023-10-06T13:54:42.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c803c640 15 client.4453.objecter handle_osd_op_reply completed tid 20 2023-10-06T13:54:42.414 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c803c640 15 client.4453.objecter _finish_op 20 2023-10-06T13:54:42.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c803c640 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 4 2023-10-06T13:54:42.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c803c640 15 client.4453.objecter _session_op_remove 6 20 2023-10-06T13:54:42.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c803c640 5 client.4453.objecter 0 in flight 2023-10-06T13:54:42.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c803c640 20 librados: complete_aio_write 0x55d592a81840 2023-10-06T13:54:42.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (22 ms) 2023-10-06T13:54:42.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 10 librados: watch_flush enter 2023-10-06T13:54:42.415 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 10 librados: watch_flush exit 2023-10-06T13:54:42.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 10 client.4453.objecter close_session for osd.6 2023-10-06T13:54:42.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x55d592aa8880 msgr2=0x55d592b501b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:54:42.416 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 1 --2- 172.21.15.19:0/880871861 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x55d592aa8880 0x55d592b501b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T13:54:42.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 20 client.4453.objecter put_session s=0x55d592927d10 osd=6 1 2023-10-06T13:54:42.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 20 client.4453.objecter shutdown clearing up homeless session... 2023-10-06T13:54:42.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 10 client.4453.objecter successfully canceled tick 2023-10-06T13:54:42.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f55b406a400 msgr2=0x7f55b406c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:54:42.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 1 --2- 172.21.15.19:0/880871861 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f55b406a400 0x7f55b406c8b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55bc055000 tx=0x7f55bc04d000 comp rx=0 tx=0).stop 2023-10-06T13:54:42.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 10 monclient: shutdown 2023-10-06T13:54:42.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T13:54:42.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5929d30c0 msgr2=0x55d592aa0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T13:54:42.417 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 1 --2- 172.21.15.19:0/880871861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5929d30c0 0x55d592aa0bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f55b8013cb0 tx=0x7f55b8002fb0 comp rx=0 tx=0).stop 2023-10-06T13:54:42.418 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c803c640 1 -- 172.21.15.19:0/880871861 reap_dead start 2023-10-06T13:54:42.418 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 shutdown_connections 2023-10-06T13:54:42.418 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 >> 172.21.15.19:0/880871861 conn(0x55d592a182a0 msgr2=0x55d5929d9740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T13:54:42.418 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 shutdown_connections 2023-10-06T13:54:42.418 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.410+0000 7f55c9d3c780 1 -- 172.21.15.19:0/880871861 wait complete. 2023-10-06T13:54:42.418 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.411+0000 7f55c9d3c780 1 librados: shutdown 2023-10-06T13:54:42.418 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [----------] 12 tests from AsioRados (18217 ms total) 2023-10-06T13:54:42.418 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: 2023-10-06T13:54:42.418 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [----------] Global test environment tear-down 2023-10-06T13:54:42.419 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (28166 ms total) 2023-10-06T13:54:42.419 INFO:tasks.workunit.client.0.smithi019.stdout: api_asio: [ PASSED ] 12 tests. 2023-10-06T13:54:43.472 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:02:46.594 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:43.474 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2023-10-06T13:54:43.474 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [----------] Global test environment set-up. 2023-10-06T13:54:43.474 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [----------] 4 tests from LibRadosService 2023-10-06T13:54:43.474 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2023-10-06T13:54:43.474 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5372 ms) 2023-10-06T13:54:43.474 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2023-10-06T13:54:43.474 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [ OK ] LibRadosService.RegisterLate (473 ms) 2023-10-06T13:54:43.475 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2023-10-06T13:54:43.475 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: cluster: 2023-10-06T13:54:43.475 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: id: c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:54:43.475 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: health: HEALTH_OK 2023-10-06T13:54:43.475 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 2023-10-06T13:54:43.475 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: services: 2023-10-06T13:54:43.475 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 2m) 2023-10-06T13:54:43.475 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: mgr: x(active, since 116s), standbys: y 2023-10-06T13:54:43.475 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: osd: 8 osds: 8 up (since 73s), 8 in (since 116s) 2023-10-06T13:54:43.475 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: laundry: 2 daemons active (1 hosts) 2023-10-06T13:54:43.475 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 2023-10-06T13:54:43.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: data: 2023-10-06T13:54:43.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: pools: 26 pools, 777 pgs 2023-10-06T13:54:43.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: objects: 2 objects, 449 KiB 2023-10-06T13:54:43.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: usage: 213 MiB used, 720 GiB / 720 GiB avail 2023-10-06T13:54:43.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: pgs: 94.595% pgs unknown 2023-10-06T13:54:43.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 4.247% pgs not active 2023-10-06T13:54:43.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 735 unknown 2023-10-06T13:54:43.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 33 creating+peering 2023-10-06T13:54:43.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 9 active+clean 2023-10-06T13:54:43.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 2023-10-06T13:54:43.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 2023-10-06T13:54:43.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: cluster: 2023-10-06T13:54:43.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: id: c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T13:54:43.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: health: HEALTH_OK 2023-10-06T13:54:43.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 2023-10-06T13:54:43.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: services: 2023-10-06T13:54:43.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 2m) 2023-10-06T13:54:43.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: mgr: x(active, since 118s), standbys: y 2023-10-06T13:54:43.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: osd: 8 osds: 8 up (since 75s), 8 in (since 118s) 2023-10-06T13:54:43.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2023-10-06T13:54:43.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: laundry: 1 daemon active (1 hosts) 2023-10-06T13:54:43.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 2023-10-06T13:54:43.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: data: 2023-10-06T13:54:43.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: pools: 28 pools, 817 pgs 2023-10-06T13:54:43.478 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: objects: 2 objects, 449 KiB 2023-10-06T13:54:43.478 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: usage: 213 MiB used, 720 GiB / 720 GiB avail 2023-10-06T13:54:43.478 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: pgs: 74.296% pgs unknown 2023-10-06T13:54:43.478 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 24.235% pgs not active 2023-10-06T13:54:43.478 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 607 unknown 2023-10-06T13:54:43.478 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 175 creating+peering 2023-10-06T13:54:43.478 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 23 creating+activating 2023-10-06T13:54:43.478 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 11 active+clean 2023-10-06T13:54:43.478 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 1 active 2023-10-06T13:54:43.478 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 2023-10-06T13:54:43.478 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 2023-10-06T13:54:43.479 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [ OK ] LibRadosService.StatusFormat (3078 ms) 2023-10-06T13:54:43.479 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [ RUN ] LibRadosService.Status 2023-10-06T13:54:43.479 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [ OK ] LibRadosService.Status (20066 ms) 2023-10-06T13:54:43.479 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [----------] 4 tests from LibRadosService (28989 ms total) 2023-10-06T13:54:43.479 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: 2023-10-06T13:54:43.479 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [----------] Global test environment tear-down 2023-10-06T13:54:43.479 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [==========] 4 tests from 1 test suite ran. (28989 ms total) 2023-10-06T13:54:43.479 INFO:tasks.workunit.client.0.smithi019.stdout: api_service: [ PASSED ] 4 tests. 2023-10-06T13:54:43.479 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:02:46.598 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:43.514 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:02:46.637 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:43.523 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:02:46.620 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:43.526 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:02:46.623 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:43.528 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:02:46.651 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:43.617 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:02:46.714 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:43.654 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:02:46.777 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:43.708 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:02:46.831 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:54:45.852 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2023-10-06T13:54:45.853 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [----------] Global test environment set-up. 2023-10-06T13:54:45.853 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2023-10-06T13:54:45.853 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2023-10-06T13:54:45.853 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5430 ms) 2023-10-06T13:54:45.853 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2023-10-06T13:54:45.853 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (436 ms) 2023-10-06T13:54:45.853 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2023-10-06T13:54:45.853 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20288 ms) 2023-10-06T13:54:45.853 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2023-10-06T13:54:45.853 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: attempt 0 of 20 2023-10-06T13:54:45.854 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: failed to deregister: 2023-10-06T13:54:45.854 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: { 2023-10-06T13:54:45.854 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "epoch": 11, 2023-10-06T13:54:45.854 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "modified": "2023-10-06T13:54:41.274242+0000", 2023-10-06T13:54:45.854 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "services": { 2023-10-06T13:54:45.854 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "laundry": { 2023-10-06T13:54:45.854 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "daemons": { 2023-10-06T13:54:45.854 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "summary": "", 2023-10-06T13:54:45.854 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "close-test-pid92333": { 2023-10-06T13:54:45.854 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "start_epoch": 11, 2023-10-06T13:54:45.854 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "start_stamp": "2023-10-06T13:54:40.709875+0000", 2023-10-06T13:54:45.855 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "gid": 4617, 2023-10-06T13:54:45.855 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "addr": "172.21.15.19:0/4124662382", 2023-10-06T13:54:45.855 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "metadata": { 2023-10-06T13:54:45.855 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "arch": "x86_64", 2023-10-06T13:54:45.855 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "ceph_release": "reef", 2023-10-06T13:54:45.855 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "ceph_version": "ceph version 18.2.0-596-g4af54c4a (4af54c4a1461e3792e9783e337bb62ce429214f3) reef (stable)", 2023-10-06T13:54:45.855 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "ceph_version_short": "18.2.0-596-g4af54c4a", 2023-10-06T13:54:45.855 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2023-10-06T13:54:45.855 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "distro": "centos", 2023-10-06T13:54:45.855 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2023-10-06T13:54:45.855 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "distro_version": "9", 2023-10-06T13:54:45.856 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "foo": "bar", 2023-10-06T13:54:45.856 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "hostname": "smithi019", 2023-10-06T13:54:45.856 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Oct 5 02:38:42 UTC 2023", 2023-10-06T13:54:45.856 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "kernel_version": "5.14.0-373.el9.x86_64", 2023-10-06T13:54:45.856 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "mem_swap_kb": "0", 2023-10-06T13:54:45.856 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "mem_total_kb": "32482648", 2023-10-06T13:54:45.856 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "os": "Linux", 2023-10-06T13:54:45.856 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "this": "that" 2023-10-06T13:54:45.856 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: }, 2023-10-06T13:54:45.856 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "task_status": {} 2023-10-06T13:54:45.856 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: }, 2023-10-06T13:54:45.856 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "pid92309": { 2023-10-06T13:54:45.857 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "start_epoch": 10, 2023-10-06T13:54:45.857 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "start_stamp": "2023-10-06T13:54:23.483353+0000", 2023-10-06T13:54:45.857 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "gid": 4657, 2023-10-06T13:54:45.857 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "addr": "172.21.15.19:0/1261881877", 2023-10-06T13:54:45.857 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "metadata": { 2023-10-06T13:54:45.857 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "arch": "x86_64", 2023-10-06T13:54:45.857 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "ceph_release": "reef", 2023-10-06T13:54:45.857 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "ceph_version": "ceph version 18.2.0-596-g4af54c4a (4af54c4a1461e3792e9783e337bb62ce429214f3) reef (stable)", 2023-10-06T13:54:45.857 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "ceph_version_short": "18.2.0-596-g4af54c4a", 2023-10-06T13:54:45.857 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2023-10-06T13:54:46.738 INFO:tasks.ceph.osd.5.smithi083.stderr:2023-10-06T13:54:45.858+0000 13263640 -1 bdev(0x16eab920 /var/lib/ceph/osd/ceph-5/block) debug_aio_unlink stalled aio 0x17a50da0 since 2023-10-06T13:54:39.840638+0000, timeout is 5s 2023-10-06T13:54:48.716 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:54:48.705+0000 10a5e640 -1 osd.4 48 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4522.0:2 22.1a 22:5c6776db:::0.obj:head [write 0~59 in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) 2023-10-06T13:54:50.072 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:54:49.996+0000 10a5e640 -1 osd.4 49 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4522.0:2 22.1a 22:5c6776db:::0.obj:head [write 0~59 in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) 2023-10-06T13:54:51.064 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "distro": "centos", 2023-10-06T13:54:51.064 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2023-10-06T13:54:51.064 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "distro_version": "9", 2023-10-06T13:54:51.064 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "foo": "bar", 2023-10-06T13:54:51.065 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "hostname": "smithi019", 2023-10-06T13:54:51.065 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Oct 5 02:38:42 UTC 2023", 2023-10-06T13:54:51.065 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "kernel_version": "5.14.0-373.el9.x86_64", 2023-10-06T13:54:51.065 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "mem_swap_kb": "0", 2023-10-06T13:54:51.065 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "mem_total_kb": "32482648", 2023-10-06T13:54:51.065 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "os": "Linux", 2023-10-06T13:54:51.065 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "this": "that" 2023-10-06T13:54:51.065 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: }, 2023-10-06T13:54:51.065 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: "task_status": {} 2023-10-06T13:54:51.065 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: } 2023-10-06T13:54:51.065 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: } 2023-10-06T13:54:51.065 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: } 2023-10-06T13:54:51.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: } 2023-10-06T13:54:51.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: } 2023-10-06T13:54:51.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: 2023-10-06T13:54:51.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: attempt 1 of 20 2023-10-06T13:54:51.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (10432 ms) 2023-10-06T13:54:51.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (36587 ms total) 2023-10-06T13:54:51.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: 2023-10-06T13:54:51.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [----------] Global test environment tear-down 2023-10-06T13:54:51.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (36588 ms total) 2023-10-06T13:54:51.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_service_pp: [ PASSED ] 4 tests. 2023-10-06T13:54:51.104 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:54:50.987+0000 10a5e640 -1 osd.4 50 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4494.0:2 25.a 25.4061870a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e26) 2023-10-06T13:54:51.484 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T13:54:51.476+0000 1125f640 -1 osd.1 50 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4491.0:1 29.4s0 29:391d6802:test-rados-api-smithi019-92135-2::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) 2023-10-06T13:54:52.178 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:54:52.172+0000 10a5e640 -1 osd.4 51 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4494.0:2 25.a 25.4061870a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e26) 2023-10-06T13:54:53.391 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:54:53.388+0000 10a5e640 -1 osd.4 52 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4494.0:2 25.a 25.4061870a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e26) 2023-10-06T13:54:54.444 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:54:54.439+0000 10a5e640 -1 osd.4 53 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4494.0:2 25.a 25.4061870a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e26) 2023-10-06T13:54:55.524 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:54:55.487+0000 10a5e640 -1 osd.4 54 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4494.0:2 25.a 25.4061870a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e26) 2023-10-06T13:54:56.685 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:54:56.600+0000 10a5e640 -1 osd.4 55 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4494.0:2 25.a 25.4061870a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e26) 2023-10-06T13:54:57.669 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:54:57.641+0000 10a5e640 -1 osd.4 56 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4494.0:2 25.a 25.4061870a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e26) 2023-10-06T13:54:58.721 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:54:58.633+0000 10a5e640 -1 osd.4 57 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4494.0:2 25.a 25.4061870a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e26) 2023-10-06T13:55:00.152 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:55:00.133+0000 10a5e640 -1 osd.4 58 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4494.0:2 25.a 25:50e18602:test-rados-api-smithi019-92175-2::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26) 2023-10-06T13:55:01.457 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:55:01.443+0000 10a5e640 -1 osd.4 59 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4494.0:2 25.a 25:50e18602:test-rados-api-smithi019-92175-2::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26) 2023-10-06T13:55:02.572 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T13:55:02.478+0000 10a5e640 -1 osd.4 60 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4494.0:2 25.a 25:50e18602:test-rados-api-smithi019-92175-2::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26) 2023-10-06T13:55:07.249 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:03:10.372 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:55:07.251 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:03:10.374 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:55:07.643 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:03:10.766 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:55:08.130 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:03:11.227 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:55:08.136 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:03:11.233 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:55:08.362 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:03:11.459 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:55:08.534 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:03:11.657 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:55:08.548 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:03:11.671 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:55:08.798 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:03:11.921 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:55:10.902 INFO:tasks.workunit.client.0.smithi019.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2023-10-06T13:55:10.902 INFO:tasks.workunit.client.0.smithi019.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2023-10-06T13:55:10.902 INFO:tasks.workunit.client.0.smithi019.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2023-10-06T13:55:10.902 INFO:tasks.workunit.client.0.smithi019.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2023-10-06T13:55:10.902 INFO:tasks.workunit.client.0.smithi019.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (56368 ms) 2023-10-06T13:55:10.902 INFO:tasks.workunit.client.0.smithi019.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (56368 ms total) 2023-10-06T13:55:10.902 INFO:tasks.workunit.client.0.smithi019.stdout: api_cls_remote_reads: 2023-10-06T13:55:10.902 INFO:tasks.workunit.client.0.smithi019.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2023-10-06T13:55:10.903 INFO:tasks.workunit.client.0.smithi019.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (56368 ms total) 2023-10-06T13:55:10.903 INFO:tasks.workunit.client.0.smithi019.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2023-10-06T13:55:17.402 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: Running main() from gmock_main.cc 2023-10-06T13:55:17.403 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2023-10-06T13:55:17.403 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: [----------] Global test environment set-up. 2023-10-06T13:55:17.403 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2023-10-06T13:55:17.403 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2023-10-06T13:55:17.403 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (1088 ms) 2023-10-06T13:55:17.403 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2023-10-06T13:55:17.403 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (83 ms) 2023-10-06T13:55:17.403 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2023-10-06T13:55:17.403 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (61834 ms) 2023-10-06T13:55:17.403 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (63005 ms total) 2023-10-06T13:55:17.403 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: 2023-10-06T13:55:17.404 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: [----------] Global test environment tear-down 2023-10-06T13:55:17.404 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (63005 ms total) 2023-10-06T13:55:17.404 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2023-10-06T13:55:20.804 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: Running main() from gmock_main.cc 2023-10-06T13:55:20.804 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2023-10-06T13:55:20.804 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [----------] Global test environment set-up. 2023-10-06T13:55:20.804 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [----------] 9 tests from LibRadosPools 2023-10-06T13:55:20.804 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2023-10-06T13:55:20.804 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ OK ] LibRadosPools.PoolList (6067 ms) 2023-10-06T13:55:20.805 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2023-10-06T13:55:20.805 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (5741 ms) 2023-10-06T13:55:20.805 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2023-10-06T13:55:20.805 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (4408 ms) 2023-10-06T13:55:20.805 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2023-10-06T13:55:20.805 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3367 ms) 2023-10-06T13:55:20.805 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2023-10-06T13:55:20.805 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3694 ms) 2023-10-06T13:55:20.805 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2023-10-06T13:55:20.806 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (6907 ms) 2023-10-06T13:55:20.806 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2023-10-06T13:55:20.806 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5654 ms) 2023-10-06T13:55:20.806 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2023-10-06T13:55:20.806 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5607 ms) 2023-10-06T13:55:20.806 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2023-10-06T13:55:20.806 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (25078 ms) 2023-10-06T13:55:20.806 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [----------] 9 tests from LibRadosPools (66523 ms total) 2023-10-06T13:55:20.806 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: 2023-10-06T13:55:20.806 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [----------] Global test environment tear-down 2023-10-06T13:55:20.807 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (66523 ms total) 2023-10-06T13:55:20.807 INFO:tasks.workunit.client.0.smithi019.stdout: api_pool: [ PASSED ] 9 tests. 2023-10-06T13:55:25.984 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: Running main() from gmock_main.cc 2023-10-06T13:55:25.985 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2023-10-06T13:55:25.985 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [----------] Global test environment set-up. 2023-10-06T13:55:25.985 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2023-10-06T13:55:25.985 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2023-10-06T13:55:25.985 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (1257 ms) 2023-10-06T13:55:25.985 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2023-10-06T13:55:25.985 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (79 ms) 2023-10-06T13:55:25.985 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2023-10-06T13:55:25.985 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (58049 ms) 2023-10-06T13:55:25.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2023-10-06T13:55:25.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:08.186922+0000 mon.a [INF] Health check cleared: SLOW_OPS (was: 2 slow ops, oldest one blocked for 40 sec, osd.4 has slow ops) 2023-10-06T13:55:25.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:09.319166+0000 mon.a [INF] from='client.? 172.21.15.19:0/3298220227' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi019-91709-4","app": "rados","yes_i_really_mean_it": true}]': finished 2023-10-06T13:55:25.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:10.566646+0000 mon.a [INF] from='client.? 172.21.15.19:0/1049492116' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi019-92135-1"}]: dispatch 2023-10-06T13:55:25.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:11.793353+0000 mon.a [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T13:55:25.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:13.467561+0000 mon.c [INF] from='client.? 172.21.15.19:0/2784380378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi019-92122-7"}]: dispatch 2023-10-06T13:55:25.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:14.847030+0000 mon.a [WRN] Health check failed: Slow OSD heartbeats on back (longest 13294.580ms) (OSD_SLOW_PING_TIME_BACK) 2023-10-06T13:55:25.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:14.858692+0000 mon.a [WRN] Health check failed: Slow OSD heartbeats on front (longest 14576.165ms) (OSD_SLOW_PING_TIME_FRONT) 2023-10-06T13:55:25.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:15.404575+0000 mon.a [INF] from='client.5932 172.21.15.19:0/3982300358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi019-91688-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:25.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:15.585541+0000 mon.a [INF] from='client.? 172.21.15.19:0/1049492116' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi019-92135-1"}]': finished 2023-10-06T13:55:25.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:15.722589+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi019-92122-7"}]: dispatch 2023-10-06T13:55:25.986 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:16.151699+0000 mon.a [INF] from='client.? 172.21.15.19:0/1049492116' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi019-92135-1"}]: dispatch 2023-10-06T13:55:25.987 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:16.153643+0000 mon.b [INF] from='client.? 172.21.15.19:0/2715875376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi019-91726-16"}]: dispatch 2023-10-06T13:55:25.987 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:17.194781+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi019-91726-16"}]: dispatch 2023-10-06T13:55:25.987 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:17.282684+0000 mon.a [INF] from='client.? 172.21.15.19:0/3802078807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi019-92074-7"}]: dispatch 2023-10-06T13:55:28.524 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:16.955503+0000 mon.c [INF] from='client.? 172.21.15.19:0/2784380378' entit api_c_read_operations: Running main() from gmock_main.cc 2023-10-06T13:55:28.524 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2023-10-06T13:55:28.524 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [----------] Global test environment set-up. 2023-10-06T13:55:28.524 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2023-10-06T13:55:28.524 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2023-10-06T13:55:28.525 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2023-10-06T13:55:28.525 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2023-10-06T13:55:28.525 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (44799 ms) 2023-10-06T13:55:28.525 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2023-10-06T13:55:28.525 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (178 ms) 2023-10-06T13:55:28.525 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2023-10-06T13:55:28.525 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (4253 ms) 2023-10-06T13:55:28.525 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2023-10-06T13:55:28.525 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (521 ms) 2023-10-06T13:55:28.525 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2023-10-06T13:55:28.525 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (245 ms) 2023-10-06T13:55:28.525 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2023-10-06T13:55:28.525 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (167 ms) 2023-10-06T13:55:28.526 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2023-10-06T13:55:28.526 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (188 ms) 2023-10-06T13:55:28.526 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2023-10-06T13:55:28.526 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (138 ms) 2023-10-06T13:55:28.526 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2023-10-06T13:55:28.526 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (71 ms) 2023-10-06T13:55:28.526 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2023-10-06T13:55:28.526 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (180 ms) 2023-10-06T13:55:28.526 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2023-10-06T13:55:28.526 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (225 ms) 2023-10-06T13:55:28.526 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2023-10-06T13:55:28.526 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (91 ms) 2023-10-06T13:55:28.527 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2023-10-06T13:55:28.527 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (5686 ms) 2023-10-06T13:55:28.527 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2023-10-06T13:55:28.527 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (2625 ms) 2023-10-06T13:55:28.527 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2023-10-06T13:55:28.527 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (571 ms) 2023-10-06T13:55:28.527 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2023-10-06T13:55:28.527 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (93 ms) 2023-10-06T13:55:28.527 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (60031 ms total) 2023-10-06T13:55:28.527 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: 2023-10-06T13:55:28.527 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [----------] Global test environment tear-down 2023-10-06T13:55:28.527 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (74028 ms total) 2023-10-06T13:55:28.527 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2023-10-06T13:55:31.739 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2023-10-06T13:55:31.739 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [----------] Global test environment set-up. 2023-10-06T13:55:31.739 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [----------] 7 tests from LibRadosList 2023-10-06T13:55:31.739 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2023-10-06T13:55:31.739 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ OK ] LibRadosList.ListObjects (49650 ms) 2023-10-06T13:55:31.740 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2023-10-06T13:55:31.740 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (1143 ms) 2023-10-06T13:55:31.740 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2023-10-06T13:55:31.740 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: myset foo1,foo2,foo3 2023-10-06T13:55:31.740 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo1 2023-10-06T13:55:31.740 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo2 2023-10-06T13:55:31.740 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo3 2023-10-06T13:55:31.740 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: myset foo1,foo4,foo5 2023-10-06T13:55:31.740 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo4 2023-10-06T13:55:31.740 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo5 2023-10-06T13:55:31.740 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo1 2023-10-06T13:55:31.740 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: myset foo6,foo7 2023-10-06T13:55:31.740 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo7 2023-10-06T13:55:31.741 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo6 2023-10-06T13:55:31.741 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2023-10-06T13:55:31.741 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: ns1:foo4 2023-10-06T13:55:31.741 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: ns1:foo5 2023-10-06T13:55:31.741 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: ns2:foo7 2023-10-06T13:55:31.741 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: ns2:foo6 2023-10-06T13:55:31.741 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: ns1:foo1 2023-10-06T13:55:31.741 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: :foo1 2023-10-06T13:55:31.741 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: :foo2 2023-10-06T13:55:31.741 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: :foo3 2023-10-06T13:55:31.741 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (10218 ms) 2023-10-06T13:55:31.741 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2023-10-06T13:55:31.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 1 0 2023-10-06T13:55:31.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 10 0 2023-10-06T13:55:31.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 13 0 2023-10-06T13:55:31.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 7 0 2023-10-06T13:55:31.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 14 0 2023-10-06T13:55:31.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 0 0 2023-10-06T13:55:31.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 15 0 2023-10-06T13:55:31.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 11 0 2023-10-06T13:55:31.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 5 0 2023-10-06T13:55:31.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 8 0 2023-10-06T13:55:31.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 6 0 2023-10-06T13:55:31.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 3 0 2023-10-06T13:55:31.742 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 4 0 2023-10-06T13:55:31.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 12 0 2023-10-06T13:55:31.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 9 0 2023-10-06T13:55:31.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 2 0 2023-10-06T13:55:31.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2023-10-06T13:55:31.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (9509 ms) 2023-10-06T13:55:31.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2023-10-06T13:55:31.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: x cursor=MIN 2023-10-06T13:55:31.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=1 cursor=10:02547ec2:::1:head 2023-10-06T13:55:31.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=10 cursor=10:52ea6a34:::10:head 2023-10-06T13:55:31.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=13 cursor=10:566253c9:::13:head 2023-10-06T13:55:31.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=7 cursor=10:5c6b0b28:::7:head 2023-10-06T13:55:31.743 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=14 cursor=10:62a1935d:::14:head 2023-10-06T13:55:31.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=0 cursor=10:6cac518f:::0:head 2023-10-06T13:55:31.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=15 cursor=10:863748b0:::15:head 2023-10-06T13:55:31.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=11 cursor=10:89d3ae78:::11:head 2023-10-06T13:55:31.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=5 cursor=10:b29083e3:::5:head 2023-10-06T13:55:31.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=8 cursor=10:bd63b0f1:::8:head 2023-10-06T13:55:31.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=6 cursor=10:c4fdafeb:::6:head 2023-10-06T13:55:31.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=3 cursor=10:cfc208b3:::3:head 2023-10-06T13:55:31.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=4 cursor=10:d83876eb:::4:head 2023-10-06T13:55:31.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=12 cursor=10:de5d7c5f:::12:head 2023-10-06T13:55:31.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=9 cursor=10:e960b815:::9:head 2023-10-06T13:55:31.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > oid=2 cursor=10:f905c69b:::2:head 2023-10-06T13:55:31.744 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: FIRST> seek to MIN oid=1 2023-10-06T13:55:31.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=1 cursor=10:02547ec2:::1:head 2023-10-06T13:55:31.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:02547ec2:::1:head 2023-10-06T13:55:31.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:02547ec2:::1:head -> 1 2023-10-06T13:55:31.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=10 cursor=10:52ea6a34:::10:head 2023-10-06T13:55:31.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:52ea6a34:::10:head 2023-10-06T13:55:31.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:52ea6a34:::10:head -> 10 2023-10-06T13:55:31.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=13 cursor=10:566253c9:::13:head 2023-10-06T13:55:31.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:566253c9:::13:head 2023-10-06T13:55:31.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:566253c9:::13:head -> 13 2023-10-06T13:55:31.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=7 cursor=10:5c6b0b28:::7:head 2023-10-06T13:55:31.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:5c6b0b28:::7:head 2023-10-06T13:55:31.745 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:5c6b0b28:::7:head -> 7 2023-10-06T13:55:34.673 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=14 cursor=10:62a1935d:::14:head 2023-10-06T13:55:34.673 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:62a1935d:::14:head 2023-10-06T13:55:34.673 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:62a1935d:::14:head -> 14 2023-10-06T13:55:34.673 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=0 cursor=10:6cac518f:::0:head 2023-10-06T13:55:34.673 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:6cac518f:::0:head 2023-10-06T13:55:34.673 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:6cac518f:::0:head -> 0 2023-10-06T13:55:34.673 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=15 cursor=10:863748b0:::15:head 2023-10-06T13:55:34.673 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:863748b0:::15:head 2023-10-06T13:55:34.673 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:863748b0:::15:head -> 15 2023-10-06T13:55:34.674 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=11 cursor=10:89d3ae78:::11:head 2023-10-06T13:55:34.674 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:89d3ae78:::11:head 2023-10-06T13:55:34.674 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:89d3ae78:::11:head -> 11 2023-10-06T13:55:34.674 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=5 cursor=10:b29083e3:::5:head 2023-10-06T13:55:34.674 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:b29083e3:::5:head 2023-10-06T13:55:34.674 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:b29083e3:::5:head -> 5 2023-10-06T13:55:34.674 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=8 cursor=10:bd63b0f1:::8:head 2023-10-06T13:55:34.674 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:bd63b0f1:::8:head 2023-10-06T13:55:34.674 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:bd63b0f1:::8:head -> 8 2023-10-06T13:55:34.674 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=6 cursor=10:c4fdafeb:::6:head 2023-10-06T13:55:34.674 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:c4fdafeb:::6:head 2023-10-06T13:55:34.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:c4fdafeb:::6:head -> 6 2023-10-06T13:55:34.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=3 cursor=10:cfc208b3:::3:head 2023-10-06T13:55:34.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:cfc208b3:::3:head 2023-10-06T13:55:34.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:cfc208b3:::3:head -> 3 2023-10-06T13:55:34.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=4 cursor=10:d83876eb:::4:head 2023-10-06T13:55:34.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:d83876eb:::4:head 2023-10-06T13:55:34.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:d83876eb:::4:head -> 4 2023-10-06T13:55:34.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=12 cursor=10:de5d7c5f:::12:head 2023-10-06T13:55:34.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:de5d7c5f:::12:head 2023-10-06T13:55:34.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:de5d7c5f:::12:head -> 12 2023-10-06T13:55:34.675 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=9 cursor=10:e960b815:::9:head 2023-10-06T13:55:34.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:e960b815:::9:head 2023-10-06T13:55:34.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:e960b815:::9:head -> 9 2023-10-06T13:55:34.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : oid=2 cursor=10:f905c69b:::2:head 2023-10-06T13:55:34.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:f905c69b:::2:head 2023-10-06T13:55:34.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:f905c69b:::2:head -> 2 2023-10-06T13:55:34.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:52ea6a34:::10:head 2023-10-06T13:55:34.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:52ea6a34:::10:head expected=10:52ea6a34:::10:head 2023-10-06T13:55:34.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:52ea6a34:::10:head -> 10 2023-10-06T13:55:34.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=10 expected=10 2023-10-06T13:55:34.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:cfc208b3:::3:head 2023-10-06T13:55:34.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:cfc208b3:::3:head expected=10:cfc208b3:::3:head 2023-10-06T13:55:34.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:cfc208b3:::3:head -> 3 2023-10-06T13:55:34.676 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=3 expected=3 2023-10-06T13:55:34.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:d83876eb:::4:head 2023-10-06T13:55:34.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:d83876eb:::4:head expected=10:d83876eb:::4:head 2023-10-06T13:55:34.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:d83876eb:::4:head -> 4 2023-10-06T13:55:34.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=4 expected=4 2023-10-06T13:55:34.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:c4fdafeb:::6:head 2023-10-06T13:55:34.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:c4fdafeb:::6:head expected=10:c4fdafeb:::6:head 2023-10-06T13:55:34.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:c4fdafeb:::6:head -> 6 2023-10-06T13:55:34.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=6 expected=6 2023-10-06T13:55:34.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:62a1935d:::14:head 2023-10-06T13:55:34.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:62a1935d:::14:head expected=10:62a1935d:::14:head 2023-10-06T13:55:34.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:62a1935d:::14:head -> 14 2023-10-06T13:55:34.677 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=14 expected=14 2023-10-06T13:55:34.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:f905c69b:::2:head 2023-10-06T13:55:34.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:f905c69b:::2:head expected=10:f905c69b:::2:head 2023-10-06T13:55:34.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:f905c69b:::2:head -> 2 2023-10-06T13:55:34.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=2 expected=2 2023-10-06T13:55:34.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:e960b815:::9:head 2023-10-06T13:55:34.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:e960b815:::9:head expected=10:e960b815:::9:head 2023-10-06T13:55:34.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:e960b815:::9:head -> 9 2023-10-06T13:55:34.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=9 expected=9 2023-10-06T13:55:34.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:de5d7c5f:::12:head 2023-10-06T13:55:34.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:de5d7c5f:::12:head expected=10:de5d7c5f:::12:head 2023-10-06T13:55:34.678 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:de5d7c5f:::12:head -> 12 2023-10-06T13:55:38.217 INFO:tasks.workunit.client.0.smithi019.stdout: api_liy='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi019-92122-7"}]: dispatch 2023-10-06T13:55:38.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:16.977634+0000 mon.c [INF] from='client.? 172.21.15.19:0/1449051461' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2023-10-06T13:55:38.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:17.556515+0000 mon.b [INF] from='client.? 172.21.15.19:0/3993257607' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi019-91898-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:38.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:18.918977+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi019-91898-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:38.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:19.164776+0000 client.admin [INF] onexx 2023-10-06T13:55:38.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:19.203544+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi019-92122-7"}]: dispatch 2023-10-06T13:55:38.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:19.212269+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2023-10-06T13:55:38.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:19.456395+0000 mon.a [INF] from='client.5932 172.21.15.19:0/3982300358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi019-91688-4","app": "rados","yes_i_really_mean_it": true}]': finished 2023-10-06T13:55:38.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:19.464692+0000 mon.a [INF] from='client.? 172.21.15.19:0/1049492116' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi019-92135-1"}]': finished 2023-10-06T13:55:38.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:19.800158+0000 mon.a [INF] from='client.? 172.21.15.19:0/3802078807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi019-92074-7"}]: dispatch 2023-10-06T13:55:38.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:19.821330+0000 mon.a [INF] from='client.? 172.21.15.19:0/3802078807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi019-92074-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-10-06T13:55:38.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:19.892315+0000 mon.b [INF] from='client.? 172.21.15.19:0/586617057' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi019-91803-10"}]: dispatch 2023-10-06T13:55:38.219 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:20.742791+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi019-91803-10"}]: dispatch 2023-10-06T13:55:38.219 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:23.912547+0000 mon.b [INF] from='client.? 172.21.15.19:0/2715875376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi019-91726-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-10-06T13:55:38.219 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:23.922066+0000 mon.b [INF] from='client.? 172.21.15.19:0/586617057' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi019-91803-10"}]: dispatch 2023-10-06T13:55:38.219 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:24.203233+0000 mon.c [INF] from='client.? 172.21.15.19:0/3996323822' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi019-91877-10"}]: dispatch 2023-10-06T13:55:38.219 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:24.333452+0000 mon.b [INF] from='client.? 172.21.15.19:0/829676601' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi019-92339-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:38.219 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:25.379800+0000 mon.a [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T13:55:39.234 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:25.773685+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi019-91726-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-10-06T13:55:39.234 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:25.780058+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi019-91803-10"}]: dispatch 2023-10-06T13:55:39.234 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:25.803740+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi019-92339-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:39.234 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:26.119393+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi019-91877-10"}]: dispatch 2023-10-06T13:55:39.234 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:26.139492+0000 mon.c [INF] from='client.? 172.21.15.19:0/1449051461' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2023-10-06T13:55:39.234 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:26.217326+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi019-92122-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2023-10-06T13:55:39.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:28.299829+0000 client.admin [INF] twoxx 2023-10-06T13:55:39.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:28.342187+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2023-10-06T13:55:39.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:27.251997+0000 mon.b [INF] from='client.? 172.21.15.19:0/586617057' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi019-91803-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-10-06T13:55:39.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:27.674495+0000 mon.c [INF] from='client.? 172.21.15.19:0/2784380378' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi019-92122-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi019-92122-7"}]: dispatch 2023-10-06T13:55:39.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:27.690072+0000 mon.c [INF] from='client.? 172.21.15.19:0/3996323822' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi019-91877-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-10-06T13:55:39.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:29.837150+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi019-92122-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi019-92122-7"}]: dispatch 2023-10-06T13:55:39.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:29.842108+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi019-91877-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-10-06T13:55:39.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:31.009846+0000 mon.c [INF] from='client.? 172.21.15.19:0/942124356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi019-92175-12"}]: dispatch 2023-10-06T13:55:39.235 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:31.583759+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi019-91803-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-10-06T13:55:42.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:31.763230+0000 mon.a [INF] from='client.? 172.21.15.19:0/3802078807' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi019-92074-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi019-92074-7"}]': finished 2023-10-06T13:55:42.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:31.770436+0000 mon.a [INF] from='client.? 172.21.15.19:0/619761904' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi019-91751-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2023-10-06T13:55:42.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:31.773855+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi019-91726-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2023-10-06T13:55:42.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:31.777909+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi019-92339-3","app": "rados","yes_i_really_mean_it": true}]': finished 2023-10-06T13:55:42.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:32.899288+0000 mon.a [INF] from='client.? 172.21.15.19:0/619761904' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi019-91751-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi019-91751-22"}]: dispatch 2023-10-06T13:55:42.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:33.364530+0000 mon.b [INF] from='client.? 172.21.15.19:0/2715875376' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi019-91726-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi019-91726-16"}]: dispatch 2023-10-06T13:55:42.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:33.799395+0000 mon.b [INF] from='client.? 172.21.15.19:0/2466102689' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi019-92013-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:42.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:33.959074+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi019-92175-12"}]: dispatch 2023-10-06T13:55:42.320 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:34.110879+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2023-10-06T13:55:42.321 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:34.290308+0000 mon.c [INF] from='client.? 172.21.15.19:0/995038036' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi019-92135-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:42.321 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:34.302262+0000 mon.c [INF] from='client.? 172.21.15.19:0/1304079231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi019-91709-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:42.321 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:35.038998+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi019-91726-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi019-91726-16"}]: dispatch 2023-10-06T13:55:42.321 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:35.050536+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi019-92013-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:42.321 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:35.981761+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi019-92135-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:47.442 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:35.989845+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi019-91709-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:47.442 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:36.126640+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi019-91877-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2023-10-06T13:55:47.442 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:36.129497+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi019-91803-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2023-10-06T13:55:47.442 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:37.468421+0000 mon.b [INF] from='client.? 172.21.15.19:0/586617057' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi019-91803-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi019-91803-10"}]: dispatch 2023-10-06T13:55:47.443 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:37.786295+0000 mon.b [INF] from='client.? 172.21.15.19:0/1390863802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi019-91688-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:47.443 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:37.864569+0000 mon.c [INF] from='client.? 172.21.15.19:0/3996323822' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi019-91877-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi019-91877-10"}]: dispatch 2023-10-06T13:55:47.443 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:37.953699+0000 mon.c [INF] from='client.? 172.21.15.19:0/942124356' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi019-92175-12"}]: dispatch 2023-10-06T13:55:47.443 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:38.042694+0000 mon.c [INF] from='client.? 172.21.15.19:0/1449051461' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2023-10-06T13:55:47.443 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:38.136802+0000 mon.b [INF] from='client.? 172.21.15.19:0/1073970018' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi019-92102-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:47.443 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:38.654659+0000 mon.b [INF] from='client.? 172.21.15.19:0/3993257607' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi019-91898-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:47.443 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:38.661616+0000 mon.a [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T13:55:47.443 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:38.946450+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi019-91803-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi019-91803-10"}]: dispatch 2023-10-06T13:55:47.443 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:38.952113+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi019-91688-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:47.443 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:38.956730+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi019-92102-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:47.443 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:39.558455+0000 client.admin [INF] threexx 2023-10-06T13:55:47.443 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:39.570485+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi019-91877-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi019-91877-10"}]: dispatch 2023-10-06T13:55:56.611 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06 list_parallel: process_1_[92444]: starting. 2023-10-06T13:55:56.611 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_1_[92444]: creating pool ceph_test_rados_list_parallel.smithi019-92421 2023-10-06T13:55:56.611 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_1_[92444]: created object 0... 2023-10-06T13:55:56.611 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_1_[92444]: created object 25... 2023-10-06T13:55:56.611 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_1_[92444]: created object 49... 2023-10-06T13:55:56.611 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_1_[92444]: finishing. 2023-10-06T13:55:56.611 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_1_[92444]: shutting down. 2023-10-06T13:55:56.611 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_2_[92445]: starting. 2023-10-06T13:55:56.612 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_2_[92445]: listing objects. 2023-10-06T13:55:56.612 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_2_[92445]: listed object 0... 2023-10-06T13:55:56.612 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_2_[92445]: listed object 25... 2023-10-06T13:55:56.612 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_2_[92445]: saw 50 objects 2023-10-06T13:55:56.612 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_2_[92445]: shutting down. 2023-10-06T13:55:56.612 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:55:56.612 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_3_[107166]: starting. 2023-10-06T13:55:56.612 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_3_[107166]: creating pool ceph_test_rados_list_parallel.smithi019-92421 2023-10-06T13:55:56.612 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_3_[107166]: created object 0... 2023-10-06T13:55:56.612 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_3_[107166]: created object 25... 2023-10-06T13:55:56.612 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_3_[107166]: created object 49... 2023-10-06T13:55:56.612 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_3_[107166]: finishing. 2023-10-06T13:55:56.613 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_3_[107166]: shutting down. 2023-10-06T13:55:56.613 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:55:56.613 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_4_[107167]: starting. 2023-10-06T13:55:56.613 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_4_[107167]: listing objects. 2023-10-06T13:55:56.613 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_4_[107167]: listed object 0... 2023-10-06T13:55:56.613 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_4_[107167]: listed object 25... 2023-10-06T13:55:56.613 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_4_[107167]: saw 40 objects 2023-10-06T13:55:56.613 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_4_[107167]: shutting down. 2023-10-06T13:55:56.613 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:55:56.613 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_5_[107168]: starting. 2023-10-06T13:55:56.613 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_5_[107168]: removed 25 objects... 2023-10-06T13:55:56.613 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_5_[107168]: removed half of the objects 2023-10-06T13:55:56.614 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_5_[107168]: removed 50 objects... 2023-10-06T13:55:56.614 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_5_[107168]: removed 50 objects 2023-10-06T13:55:56.614 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_5_[107168]: shutting down. 2023-10-06T13:55:56.614 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:55:56.614 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:55:56.614 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_6_[118647]: starting. 2023-10-06T13:55:56.614 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_6_[118647]: creating pool ceph_test_rados_list_parallel.smithi019-92421 2023-10-06T13:55:56.614 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_6_[118647]: created object 0... 2023-10-06T13:55:56.614 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_6_[118647]: created object 25... 2023-10-06T13:55:56.614 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_6_[118647]: created object 49... 2023-10-06T13:55:56.614 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_6_[118647]: finishing. 2023-10-06T13:55:56.614 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_6_[118647]: shutting down. 2023-10-06T13:55:56.615 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:55:56.615 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:55:56.615 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_7_[118648]: starting. 2023-10-06T13:55:56.615 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_7_[118648]: listing objects. 2023-10-06T13:55:56.615 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_7_[118648]: listed object 0... 2023-10-06T13:55:56.615 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_7_[118648]: listed object 25... 2023-10-06T13:55:56.615 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_7_[118648]: listed object 50... 2023-10-06T13:55:56.615 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_7_[118648]: saw 57 objects 2023-10-06T13:55:56.615 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_7_[118648]: shutting down. 2023-10-06T13:55:56.615 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:55:56.615 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:55:56.615 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_8_[118649]: starting. 2023-10-06T13:55:56.615 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_8_[118649]: added 25 objects... 2023-10-06T13:55:56.616 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_8_[118649]: added half of the objects 2023-10-06T13:55:56.616 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_8_[118649]: added 50 objects... 2023-10-06T13:55:56.616 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_8_[118649]: added 50 objects 2023-10-06T13:55:56.616 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_8_[118649]: shutting down. 2023-10-06T13:55:56.616 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:55:56.616 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:55:58.084 INFO:tasks.workunit.client.0.smithi019.stdout: T13:55:39.575711+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi019-92175-12"}]: dispatch 2023-10-06T13:55:58.084 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:39.579402+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2023-10-06T13:55:58.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:39.691043+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi019-92122-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi019-92122-7"}]': finished 2023-10-06T13:55:58.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:39.714827+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi019-92013-6","app": "rados","yes_i_really_mean_it": true}]': finished 2023-10-06T13:55:58.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:39.719475+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi019-92135-4","app": "rados","yes_i_really_mean_it": true}]': finished 2023-10-06T13:55:58.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:39.722486+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi019-91709-5","app": "rados","yes_i_really_mean_it": true}]': finished 2023-10-06T13:55:58.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: got: 2023-10-06T13:55:41.220109+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi019-91898-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-10-06T13:55:58.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (44282 ms) 2023-10-06T13:55:58.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (103668 ms total) 2023-10-06T13:55:58.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: 2023-10-06T13:55:58.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [----------] Global test environment tear-down 2023-10-06T13:55:58.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (103668 ms total) 2023-10-06T13:55:58.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_cmd: [ PASSED ] 4 tests. 2023-10-06T13:55:58.843 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: Running main() from gmock_main.cc 2023-10-06T13:55:58.843 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2023-10-06T13:55:58.843 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [----------] Global test environment set-up. 2023-10-06T13:55:58.843 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2023-10-06T13:55:58.843 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: seed 91751 2023-10-06T13:55:58.843 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2023-10-06T13:55:58.843 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2023-10-06T13:55:58.843 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2023-10-06T13:55:58.843 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (10854 ms) 2023-10-06T13:55:58.843 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2023-10-06T13:55:58.844 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (13983 ms) 2023-10-06T13:55:58.844 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2023-10-06T13:55:58.844 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (3613 ms) 2023-10-06T13:55:58.844 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2023-10-06T13:55:58.844 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (3896 ms) 2023-10-06T13:55:58.844 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2023-10-06T13:55:58.844 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (37 ms) 2023-10-06T13:55:58.844 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2023-10-06T13:55:58.844 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (51 ms) 2023-10-06T13:55:58.844 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2023-10-06T13:55:58.844 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (13195 ms) 2023-10-06T13:55:58.844 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2023-10-06T13:55:58.844 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (306 ms) 2023-10-06T13:55:58.845 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2023-10-06T13:55:58.845 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (196 ms) 2023-10-06T13:55:58.845 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2023-10-06T13:55:58.845 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (66 ms) 2023-10-06T13:55:58.845 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2023-10-06T13:55:58.845 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (120 ms) 2023-10-06T13:55:58.845 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2023-10-06T13:55:58.845 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (237 ms) 2023-10-06T13:55:58.845 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2023-10-06T13:55:58.845 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (1695 ms) 2023-10-06T13:55:58.845 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2023-10-06T13:55:58.845 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (112 ms) 2023-10-06T13:55:58.846 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2023-10-06T13:55:58.846 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (303 ms) 2023-10-06T13:55:58.846 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2023-10-06T13:55:58.846 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (989 ms) 2023-10-06T13:55:58.846 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2023-10-06T13:55:58.846 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (678 ms) 2023-10-06T13:55:58.846 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2023-10-06T13:55:58.846 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (44 ms) 2023-10-06T13:55:58.846 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2023-10-06T13:55:58.846 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (250 ms) 2023-10-06T13:55:58.846 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (50626 ms total) 2023-10-06T13:55:58.847 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: 2023-10-06T13:55:58.847 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2023-10-06T13:55:58.847 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2023-10-06T13:55:58.847 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (9441 ms) 2023-10-06T13:55:58.847 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2023-10-06T13:55:58.847 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (965 ms) 2023-10-06T13:55:58.847 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2023-10-06T13:55:58.847 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (535 ms) 2023-10-06T13:55:58.847 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2023-10-06T13:55:58.847 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (350 ms) 2023-10-06T13:55:58.847 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2023-10-06T13:56:02.842 INFO:tasks.ceph.mon.c.smithi019.stderr:2023-10-06T13:56:02.766+0000 ff3d640 -1 mon.c@2(peon).paxos(paxos updating c 1..275) lease_expire from mon.0 v2:172.21.15.19:3300/0 is 0.871281624s seconds in the past; mons are probably laggy (or possibly clocks are too skewed) 2023-10-06T13:56:08.554 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 list_parallel: ******************************* 2023-10-06T13:56:08.554 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_9_[135042]: starting. 2023-10-06T13:56:08.555 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_9_[135042]: creating pool ceph_test_rados_list_parallel.smithi019-92421 2023-10-06T13:56:08.555 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_9_[135042]: created object 0... 2023-10-06T13:56:08.555 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_9_[135042]: created object 25... 2023-10-06T13:56:08.555 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_9_[135042]: created object 49... 2023-10-06T13:56:08.555 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_9_[135042]: finishing. 2023-10-06T13:56:08.555 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_9_[135042]: shutting down. 2023-10-06T13:56:08.555 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.555 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.555 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.555 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_10_[135043]: starting. 2023-10-06T13:56:08.555 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_10_[135043]: listing objects. 2023-10-06T13:56:08.556 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_10_[135043]: listed object 0... 2023-10-06T13:56:08.556 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_10_[135043]: listed object 25... 2023-10-06T13:56:08.556 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_10_[135043]: listed object 50... 2023-10-06T13:56:08.556 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_10_[135043]: listed object 75... 2023-10-06T13:56:08.556 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_10_[135043]: listed object 100... 2023-10-06T13:56:08.556 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_10_[135043]: saw 104 objects 2023-10-06T13:56:08.556 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_10_[135043]: shutting down. 2023-10-06T13:56:08.556 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.556 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.556 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.556 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_11_[135044]: starting. 2023-10-06T13:56:08.556 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_11_[135044]: added 25 objects... 2023-10-06T13:56:08.557 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_11_[135044]: added half of the objects 2023-10-06T13:56:08.557 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_11_[135044]: added 50 objects... 2023-10-06T13:56:08.557 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_11_[135044]: added 50 objects 2023-10-06T13:56:08.557 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_11_[135044]: shutting down. 2023-10-06T13:56:08.557 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.557 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.557 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.557 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_13_[135046]: starting. 2023-10-06T13:56:08.557 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_13_[135046]: removed 25 objects... 2023-10-06T13:56:08.557 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_13_[135046]: removed half of the objects 2023-10-06T13:56:08.557 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_13_[135046]: removed 50 objects... 2023-10-06T13:56:08.557 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_13_[135046]: removed 50 objects 2023-10-06T13:56:08.558 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_13_[135046]: shutting down. 2023-10-06T13:56:08.558 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.558 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.558 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.558 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_12_[135045]: starting. 2023-10-06T13:56:08.558 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_12_[135045]: added 25 objects... 2023-10-06T13:56:08.558 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_12_[135045]: added half of the objects 2023-10-06T13:56:08.558 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_12_[135045]: added 50 objects... 2023-10-06T13:56:08.558 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_12_[135045]: added 50 objects 2023-10-06T13:56:08.558 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_12_[135045]: shutting down. 2023-10-06T13:56:08.558 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.558 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.559 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.559 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.559 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_14_[146100]: starting. 2023-10-06T13:56:08.559 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_14_[146100]: creating pool ceph_test_rados_list_parallel.smithi019-92421 2023-10-06T13:56:08.559 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_14_[146100]: created object 0... 2023-10-06T13:56:08.559 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_14_[146100]: created object 25... 2023-10-06T13:56:08.559 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_14_[146100]: created object 49... 2023-10-06T13:56:08.559 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_14_[146100]: finishing. 2023-10-06T13:56:08.559 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_14_[146100]: shutting down. 2023-10-06T13:56:08.559 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.559 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.559 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.560 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:08.560 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_15_[146101]: starting. 2023-10-06T13:56:12.805 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:04:15.927 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:12.821 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:04:15.943 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:12.822 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:04:15.945 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:12.827 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:04:15.924 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:12.828 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:04:15.951 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:12.832 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:04:15.929 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:12.970 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:04:16.067 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:13.032 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:04:16.155 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:13.400 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:04:16.523 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:13.996 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_15_[146 api_stat: Running main() from gmock_main.cc 2023-10-06T13:56:13.996 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2023-10-06T13:56:13.996 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [----------] Global test environment set-up. 2023-10-06T13:56:13.996 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [----------] 5 tests from LibRadosStat 2023-10-06T13:56:13.997 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2023-10-06T13:56:13.997 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ OK ] LibRadosStat.Stat (28389 ms) 2023-10-06T13:56:13.997 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2023-10-06T13:56:13.997 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (13030 ms) 2023-10-06T13:56:13.997 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2023-10-06T13:56:13.997 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ OK ] LibRadosStat.StatNS (3142 ms) 2023-10-06T13:56:13.997 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2023-10-06T13:56:13.997 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (1045 ms) 2023-10-06T13:56:13.997 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2023-10-06T13:56:13.997 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (2222 ms) 2023-10-06T13:56:13.997 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [----------] 5 tests from LibRadosStat (47828 ms total) 2023-10-06T13:56:13.997 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: 2023-10-06T13:56:13.998 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2023-10-06T13:56:13.998 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2023-10-06T13:56:13.998 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (10520 ms) 2023-10-06T13:56:13.998 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2023-10-06T13:56:13.998 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (3726 ms) 2023-10-06T13:56:13.998 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2023-10-06T13:56:13.998 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (1216 ms) 2023-10-06T13:56:13.998 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2023-10-06T13:56:13.998 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (2261 ms) 2023-10-06T13:56:13.998 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (17723 ms total) 2023-10-06T13:56:13.998 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: 2023-10-06T13:56:13.999 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [----------] Global test environment tear-down 2023-10-06T13:56:13.999 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (119657 ms total) 2023-10-06T13:56:13.999 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat: [ PASSED ] 9 tests. 2023-10-06T13:56:14.013 INFO:tasks.workunit.client.0.smithi019.stdout:101]: listing objects. 2023-10-06T13:56:14.013 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_15_[146101]: listed object 0... 2023-10-06T13:56:14.014 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_15_[146101]: listed object 25... 2023-10-06T13:56:14.014 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_15_[146101]: listed object 50... 2023-10-06T13:56:14.014 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_15_[146101]: listed object 75... 2023-10-06T13:56:14.014 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_15_[146101]: listed object 100... 2023-10-06T13:56:14.014 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_15_[146101]: listed object 125... 2023-10-06T13:56:14.014 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_15_[146101]: saw 150 objects 2023-10-06T13:56:14.014 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_15_[146101]: shutting down. 2023-10-06T13:56:14.014 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:14.014 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:14.014 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:14.015 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:14.015 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_16_[146102]: starting. 2023-10-06T13:56:14.015 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_16_[146102]: added 25 objects... 2023-10-06T13:56:14.015 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_16_[146102]: added half of the objects 2023-10-06T13:56:14.015 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_16_[146102]: added 50 objects... 2023-10-06T13:56:14.015 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_16_[146102]: added 50 objects 2023-10-06T13:56:14.015 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: process_16_[146102]: shutting down. 2023-10-06T13:56:14.015 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:14.015 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:14.015 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:14.015 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:14.016 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******************************* 2023-10-06T13:56:14.016 INFO:tasks.workunit.client.0.smithi019.stdout: list_parallel: ******* SUCCESS ********** 2023-10-06T13:56:14.016 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_1_[92497]: starting. 2023-10-06T13:56:14.016 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_1_[92497]: creating pool ceph_test_rados_open_pools_parallel.smithi019-92469 2023-10-06T13:56:14.016 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_1_[92497]: created object 0... 2023-10-06T13:56:14.016 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_1_[92497]: created object 25... 2023-10-06T13:56:14.016 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_1_[92497]: created object 49... 2023-10-06T13:56:14.016 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_1_[92497]: finishing. 2023-10-06T13:56:14.016 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_1_[92497]: shutting down. 2023-10-06T13:56:14.016 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_2_[92498]: starting. 2023-10-06T13:56:14.016 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_2_[92498]: rados_pool_create. 2023-10-06T13:56:14.016 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_2_[92498]: rados_ioctx_create. 2023-10-06T13:56:14.017 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_2_[92498]: shutting down. 2023-10-06T13:56:14.017 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: ******************************* 2023-10-06T13:56:14.017 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_3_[112418]: starting. 2023-10-06T13:56:14.017 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_3_[112418]: creating pool ceph_test_rados_open_pools_parallel.smithi019-92469 2023-10-06T13:56:14.017 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_3_[112418]: created object 0... 2023-10-06T13:56:14.017 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_3_[112418]: created object 25... 2023-10-06T13:56:14.017 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_3_[112418]: created object 49... 2023-10-06T13:56:14.017 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_3_[112418]: finishing. 2023-10-06T13:56:14.017 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_3_[112418]: shutting down. 2023-10-06T13:56:14.017 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: ******************************* 2023-10-06T13:56:14.017 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_4_[112419]: starting. 2023-10-06T13:56:14.018 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_4_[112419]: rados_pool_create. 2023-10-06T13:56:14.018 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_4_[112419]: rados_ioctx_create. 2023-10-06T13:56:14.018 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: process_4_[112419]: shutting down. 2023-10-06T13:56:14.018 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: ******************************* 2023-10-06T13:56:14.018 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: ******************************* 2023-10-06T13:56:14.018 INFO:tasks.workunit.client.0.smithi019.stdout: open_pools_parallel: ******* SUCCESS ********** 2023-10-06T13:56:14.031 INFO:tasks.workunit.client.0.smithi019.stdout: LibRadosMiscPool 2023-10-06T13:56:14.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2023-10-06T13:56:14.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: started 0x563be5edeee0 2023-10-06T13:56:14.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: started 0x563be68e2850 2023-10-06T13:56:14.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: started 0x563be5eeabc0 2023-10-06T13:56:14.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: started 0x563be6055130 2023-10-06T13:56:14.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: started 0x563be6055bd0 2023-10-06T13:56:14.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: started 0x563be6056670 2023-10-06T13:56:14.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: started 0x563be60570b0 2023-10-06T13:56:14.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: started 0x563be6057b00 2023-10-06T13:56:14.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: started 0x563be6058530 2023-10-06T13:56:14.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: started 0x563be5fb4830 2023-10-06T13:56:14.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: started 0x563be60594b0 2023-10-06T13:56:14.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: started 0x563be60582e0 2023-10-06T13:56:14.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: started 12 aios 2023-10-06T13:56:14.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: waiting 0x563be5edeee0 2023-10-06T13:56:14.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: waiting 0x563be68e2850 2023-10-06T13:56:14.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: waiting 0x563be5eeabc0 2023-10-06T13:56:14.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: waiting 0x563be6055130 2023-10-06T13:56:14.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: waiting 0x563be6055bd0 2023-10-06T13:56:14.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: waiting 0x563be6056670 2023-10-06T13:56:14.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: waiting 0x563be60570b0 2023-10-06T13:56:14.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: waiting 0x563be6057b00 2023-10-06T13:56:14.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: waiting 0x563be6058530 2023-10-06T13:56:14.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: waiting 0x563be5fb4830 2023-10-06T13:56:14.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: waiting 0x563be60594b0 2023-10-06T13:56:14.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: waiting 0x563be60582e0 2023-10-06T13:56:14.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: done. 2023-10-06T13:56:14.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (23029 ms) 2023-10-06T13:56:14.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (23030 ms total) 2023-10-06T13:56:14.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:56:14.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2023-10-06T13:56:14.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2023-10-06T13:56:14.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2023-10-06T13:56:14.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2023-10-06T13:56:14.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ OK ] LibRadosMisc.Exec (1214 ms) 2023-10-06T13:56:14.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2023-10-06T13:56:14.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (12375 ms) 2023-10-06T13:56:14.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2023-10-06T13:56:14.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (1866 ms) 2023-10-06T13:56:14.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2023-10-06T13:56:14.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5463 ms) 2023-10-06T13:56:14.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2023-10-06T13:56:14.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2023-10-06T13:56:14.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2023-10-06T13:56:14.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2023-10-06T13:56:14.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2023-10-06T13:56:14.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (71725 ms) 2023-10-06T13:56:14.036 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [----------] 8 tests from LibRadosMisc (92643 ms total) 2023-10-06T13:56:14.036 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: 2023-10-06T13:56:14.036 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [----------] Global test environment tear-down 2023-10-06T13:56:14.036 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [==========] 11 tests from 4 test suites ran. (119733 ms total) 2023-10-06T13:56:14.036 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc: [ PASSED ] 11 tests. 2023-10-06T13:56:14.045 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_1_[92474]: starting. 2023-10-06T13:56:14.045 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_1_[92474]: creating pool ceph_test_rados_delete_pools_parallel.smithi019-92448 2023-10-06T13:56:14.045 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_1_[92474]: created object 0... 2023-10-06T13:56:14.045 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_1_[92474]: created object 25... 2023-10-06T13:56:14.045 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_1_[92474]: created object 49... 2023-10-06T13:56:14.045 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_1_[92474]: finishing. 2023-10-06T13:56:14.045 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_1_[92474]: shutting down. 2023-10-06T13:56:14.045 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_2_[92475]: starting. 2023-10-06T13:56:14.045 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_2_[92475]: deleting pool ceph_test_rados_delete_pools_parallel.smithi019-92448 2023-10-06T13:56:14.045 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_2_[92475]: shutting down. 2023-10-06T13:56:14.046 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: ******************************* 2023-10-06T13:56:14.046 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_3_[112613]: starting. 2023-10-06T13:56:14.046 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_3_[112613]: creating pool ceph_test_rados_delete_pools_parallel.smithi019-92448 2023-10-06T13:56:14.046 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_3_[112613]: created object 0... 2023-10-06T13:56:14.046 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_3_[112613]: created object 25... 2023-10-06T13:56:14.046 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_3_[112613]: created object 49... 2023-10-06T13:56:14.046 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_3_[112613]: finishing. 2023-10-06T13:56:14.046 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_3_[112613]: shutting down. 2023-10-06T13:56:14.046 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: ******************************* 2023-10-06T13:56:14.046 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_5_[112615]: starting. 2023-10-06T13:56:14.046 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_5_[112615]: listing objects. 2023-10-06T13:56:14.046 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_5_[112615]: listed object 0... 2023-10-06T13:56:14.047 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_5_[112615]: listed object 25... 2023-10-06T13:56:14.047 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_5_[112615]: saw 50 objects 2023-10-06T13:56:14.047 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_5_[112615]: shutting down. 2023-10-06T13:56:14.047 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: ******************************* 2023-10-06T13:56:14.047 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_4_[112614]: starting. 2023-10-06T13:56:14.047 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_4_[112614]: deleting pool ceph_test_rados_delete_pools_parallel.smithi019-92448 2023-10-06T13:56:14.047 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: process_4_[112614]: shutting down. 2023-10-06T13:56:14.047 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: ******************************* 2023-10-06T13:56:14.047 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: ******************************* 2023-10-06T13:56:14.047 INFO:tasks.workunit.client.0.smithi019.stdout: delete_pools_parallel: ******* SUCCESS ********** 2023-10-06T13:56:15.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: Running main() from gmock_main.cc 2023-10-06T13:56:15.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2023-10-06T13:56:15.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [----------] Global test environment set-up. 2023-10-06T13:56:15.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2023-10-06T13:56:15.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: seed 92122 2023-10-06T13:56:15.066 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2023-10-06T13:56:15.067 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (15434 ms) 2023-10-06T13:56:15.067 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2023-10-06T13:56:15.067 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (22611 ms) 2023-10-06T13:56:15.067 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2023-10-06T13:56:15.067 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (15 ms) 2023-10-06T13:56:15.067 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2023-10-06T13:56:15.067 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (1250 ms) 2023-10-06T13:56:15.067 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2023-10-06T13:56:15.067 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (5929 ms) 2023-10-06T13:56:15.067 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (45239 ms total) 2023-10-06T13:56:15.067 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: 2023-10-06T13:56:15.067 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2023-10-06T13:56:15.067 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2023-10-06T13:56:15.068 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (6675 ms) 2023-10-06T13:56:15.068 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2023-10-06T13:56:15.068 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (1758 ms) 2023-10-06T13:56:15.068 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2023-10-06T13:56:15.068 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (2227 ms) 2023-10-06T13:56:15.068 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2023-10-06T13:56:15.068 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (1534 ms) 2023-10-06T13:56:15.068 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (12194 ms total) 2023-10-06T13:56:15.068 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: 2023-10-06T13:56:15.068 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [----------] Global test environment tear-down 2023-10-06T13:56:15.068 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (120702 ms total) 2023-10-06T13:56:15.068 INFO:tasks.workunit.client.0.smithi019.stdout: api_stat_pp: [ PASSED ] 9 tests. 2023-10-06T13:56:16.099 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: Running main() from gmock_main.cc 2023-10-06T13:56:16.099 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2023-10-06T13:56:16.099 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [----------] Global test environment set-up. 2023-10-06T13:56:16.099 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2023-10-06T13:56:16.099 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: seed 91877 2023-10-06T13:56:16.099 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2023-10-06T13:56:16.100 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (15766 ms) 2023-10-06T13:56:16.100 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2023-10-06T13:56:16.100 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (19494 ms) 2023-10-06T13:56:16.100 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2023-10-06T13:56:16.100 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (6074 ms) 2023-10-06T13:56:16.100 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2023-10-06T13:56:16.100 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (2418 ms) 2023-10-06T13:56:16.100 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2023-10-06T13:56:16.100 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (1134 ms) 2023-10-06T13:56:16.100 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2023-10-06T13:56:16.100 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (1591 ms) 2023-10-06T13:56:16.100 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2023-10-06T13:56:16.101 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (2642 ms) 2023-10-06T13:56:16.101 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2023-10-06T13:56:16.101 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (289 ms) 2023-10-06T13:56:16.101 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (49408 ms total) 2023-10-06T13:56:16.101 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: 2023-10-06T13:56:16.101 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2023-10-06T13:56:16.101 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2023-10-06T13:56:16.101 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (7429 ms) 2023-10-06T13:56:16.101 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2023-10-06T13:56:16.101 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (272 ms) 2023-10-06T13:56:16.101 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2023-10-06T13:56:16.101 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1180 ms) 2023-10-06T13:56:16.101 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2023-10-06T13:56:16.102 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1235 ms) 2023-10-06T13:56:16.102 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2023-10-06T13:56:16.102 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (209 ms) 2023-10-06T13:56:16.102 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2023-10-06T13:56:16.102 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (177 ms) 2023-10-06T13:56:16.102 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2023-10-06T13:56:16.102 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (191 ms) 2023-10-06T13:56:16.102 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2023-10-06T13:56:16.102 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (342 ms) 2023-10-06T13:56:16.102 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (11035 ms total) 2023-10-06T13:56:16.102 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: 2023-10-06T13:56:16.102 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [----------] Global test environment tear-down 2023-10-06T13:56:16.103 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (121841 ms total) 2023-10-06T13:56:16.103 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock_pp: [ PASSED ] 16 tests. 2023-10-06T13:56:16.106 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: Running main() from gmock_main.cc 2023-10-06T13:56:16.106 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2023-10-06T13:56:16.106 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [----------] Global test environment set-up. 2023-10-06T13:56:16.106 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [----------] 8 tests from LibRadosLock 2023-10-06T13:56:16.106 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2023-10-06T13:56:16.107 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (35292 ms) 2023-10-06T13:56:16.107 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2023-10-06T13:56:16.107 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLock.LockShared (10602 ms) 2023-10-06T13:56:16.107 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2023-10-06T13:56:16.107 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (2111 ms) 2023-10-06T13:56:16.107 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2023-10-06T13:56:16.107 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1579 ms) 2023-10-06T13:56:16.107 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2023-10-06T13:56:16.107 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (1367 ms) 2023-10-06T13:56:16.107 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2023-10-06T13:56:16.107 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLock.Unlock (215 ms) 2023-10-06T13:56:16.107 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2023-10-06T13:56:16.108 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (400 ms) 2023-10-06T13:56:16.108 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2023-10-06T13:56:16.108 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (86 ms) 2023-10-06T13:56:16.108 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [----------] 8 tests from LibRadosLock (51654 ms total) 2023-10-06T13:56:16.108 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: 2023-10-06T13:56:16.108 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2023-10-06T13:56:16.108 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2023-10-06T13:56:16.108 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (6585 ms) 2023-10-06T13:56:16.108 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2023-10-06T13:56:16.108 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (108 ms) 2023-10-06T13:56:16.108 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2023-10-06T13:56:16.108 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1248 ms) 2023-10-06T13:56:16.109 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2023-10-06T13:56:16.109 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1227 ms) 2023-10-06T13:56:16.109 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2023-10-06T13:56:16.109 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (356 ms) 2023-10-06T13:56:16.109 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2023-10-06T13:56:16.109 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (271 ms) 2023-10-06T13:56:16.109 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2023-10-06T13:56:16.109 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (280 ms) 2023-10-06T13:56:16.109 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2023-10-06T13:56:16.109 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (214 ms) 2023-10-06T13:56:16.109 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (10289 ms total) 2023-10-06T13:56:16.109 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: 2023-10-06T13:56:16.109 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [----------] Global test environment tear-down 2023-10-06T13:56:16.110 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (121892 ms total) 2023-10-06T13:56:16.110 INFO:tasks.workunit.client.0.smithi019.stdout: api_lock: [ PASSED ] 16 tests. 2023-10-06T13:56:16.114 INFO:tasks.workunit.client.0.smithi019.stdout:(1426 ms) 2023-10-06T13:56:16.114 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2023-10-06T13:56:16.114 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (226 ms) 2023-10-06T13:56:16.114 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2023-10-06T13:56:16.114 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (208 ms) 2023-10-06T13:56:16.114 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2023-10-06T13:56:16.114 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (128 ms) 2023-10-06T13:56:16.114 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2023-10-06T13:56:16.115 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (199 ms) 2023-10-06T13:56:16.115 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2023-10-06T13:56:16.115 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (226 ms) 2023-10-06T13:56:16.115 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2023-10-06T13:56:16.115 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (269 ms) 2023-10-06T13:56:16.115 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2023-10-06T13:56:16.115 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (193 ms) 2023-10-06T13:56:16.115 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2023-10-06T13:56:16.115 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (1068 ms) 2023-10-06T13:56:16.115 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2023-10-06T13:56:16.115 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (309 ms) 2023-10-06T13:56:16.115 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2023-10-06T13:56:16.116 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (275 ms) 2023-10-06T13:56:16.116 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2023-10-06T13:56:16.116 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (134 ms) 2023-10-06T13:56:16.116 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2023-10-06T13:56:16.116 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (183 ms) 2023-10-06T13:56:16.116 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (16136 ms total) 2023-10-06T13:56:16.116 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: 2023-10-06T13:56:16.116 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [----------] Global test environment tear-down 2023-10-06T13:56:16.116 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (121921 ms total) 2023-10-06T13:56:16.116 INFO:tasks.workunit.client.0.smithi019.stdout: api_io_pp: [ PASSED ] 37 tests. 2023-10-06T13:56:16.117 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: Running main() from gmock_main.cc 2023-10-06T13:56:16.117 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2023-10-06T13:56:16.117 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [----------] Global test environment set-up. 2023-10-06T13:56:16.117 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [----------] 14 tests from LibRadosIo 2023-10-06T13:56:16.117 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2023-10-06T13:56:16.117 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (34887 ms) 2023-10-06T13:56:16.117 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2023-10-06T13:56:16.117 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2023-10-06T13:56:16.118 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2023-10-06T13:56:16.118 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: no timeout :/ 2023-10-06T13:56:16.118 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: no timeout :/ 2023-10-06T13:56:16.118 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: no timeout :/ 2023-10-06T13:56:16.118 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: no timeout :/ 2023-10-06T13:56:16.118 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: no timeout :/ 2023-10-06T13:56:16.118 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (3128 ms) 2023-10-06T13:56:16.118 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2023-10-06T13:56:16.118 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (6181 ms) 2023-10-06T13:56:16.118 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2023-10-06T13:56:16.118 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIo.Checksum (2274 ms) 2023-10-06T13:56:16.118 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2023-10-06T13:56:16.119 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (1454 ms) 2023-10-06T13:56:16.119 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2023-10-06T13:56:16.119 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (2425 ms) 2023-10-06T13:56:16.119 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2023-10-06T13:56:16.119 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (77 ms) 2023-10-06T13:56:16.119 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2023-10-06T13:56:16.119 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (74 ms) 2023-10-06T13:56:16.119 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2023-10-06T13:56:16.119 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIo.TruncTest (85 ms) 2023-10-06T13:56:16.119 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2023-10-06T13:56:16.119 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (85 ms) 2023-10-06T13:56:16.119 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2023-10-06T13:56:16.119 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (113 ms) 2023-10-06T13:56:16.120 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2023-10-06T13:56:16.120 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIo.RmXattr (411 ms) 2023-10-06T13:56:16.120 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2023-10-06T13:56:16.120 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIo.XattrIter (283 ms) 2023-10-06T13:56:16.120 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [----------] 14 tests from LibRadosIo (51477 ms total) 2023-10-06T13:56:16.120 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: 2023-10-06T13:56:16.120 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2023-10-06T13:56:16.120 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2023-10-06T13:56:16.120 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (8046 ms) 2023-10-06T13:56:16.120 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2023-10-06T13:56:16.120 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (542 ms) 2023-10-06T13:56:16.120 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2023-10-06T13:56:16.121 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (328 ms) 2023-10-06T13:56:16.121 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2023-10-06T13:56:16.121 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (557 ms) 2023-10-06T13:56:16.121 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2023-10-06T13:56:16.121 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (3291 ms) 2023-10-06T13:56:16.121 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2023-10-06T13:56:16.121 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (140 ms) 2023-10-06T13:56:16.121 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2023-10-06T13:56:16.121 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (385 ms) 2023-10-06T13:56:16.121 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2023-10-06T13:56:16.121 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (305 ms) 2023-10-06T13:56:16.121 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2023-10-06T13:56:16.121 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (663 ms) 2023-10-06T13:56:16.122 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2023-10-06T13:56:16.122 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (260 ms) 2023-10-06T13:56:16.122 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [----------] 10 tests from LibRadosIoEC (14517 ms total) 2023-10-06T13:56:16.122 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: 2023-10-06T13:56:16.122 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [----------] Global test environment tear-down 2023-10-06T13:56:16.122 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [==========] 24 tests from 2 test suites ran. (121933 ms total) 2023-10-06T13:56:16.122 INFO:tasks.workunit.client.0.smithi019.stdout: api_io: [ PASSED ] 24 tests. 2023-10-06T13:56:25.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: Running main() from gmock_main.cc 2023-10-06T13:56:25.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2023-10-06T13:56:25.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [----------] Global test environment set-up. 2023-10-06T13:56:25.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2023-10-06T13:56:25.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2023-10-06T13:56:25.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: watch_notify_test_cb 2023-10-06T13:56:25.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (45146 ms) 2023-10-06T13:56:25.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2023-10-06T13:56:25.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2023-10-06T13:56:25.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94040975728560 err -107 2023-10-06T13:56:25.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (3655 ms) 2023-10-06T13:56:25.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2023-10-06T13:56:25.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94040975740416 err -107 2023-10-06T13:56:25.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2023-10-06T13:56:25.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (351 ms) 2023-10-06T13:56:25.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2023-10-06T13:56:25.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: watch_notify2_test_cb from 4494 notify_id 266287972352 cookie 94040975740416 2023-10-06T13:56:25.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (452 ms) 2023-10-06T13:56:25.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2023-10-06T13:56:25.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: watch_notify2_test_cb from 4494 notify_id 266287972353 cookie 94040975759280 2023-10-06T13:56:25.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (478 ms) 2023-10-06T13:56:25.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2023-10-06T13:56:25.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: watch_notify2_test_cb from 4494 notify_id 266287972352 cookie 94040975792896 2023-10-06T13:56:25.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (269 ms) 2023-10-06T13:56:25.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2023-10-06T13:56:25.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: watch_notify2_test_cb from 4494 notify_id 266287972354 cookie 94040975777456 2023-10-06T13:56:25.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: watch_notify2_test_cb from 4494 notify_id 266287972354 cookie 94040975792896 2023-10-06T13:56:25.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (500 ms) 2023-10-06T13:56:25.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2023-10-06T13:56:25.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: watch_notify2_test_cb from 4494 notify_id 266287972352 cookie 94040975808720 2023-10-06T13:56:25.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: watch_notify2_test_cb from 4494 notify_id 270582939649 cookie 94040975808720 2023-10-06T13:56:25.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (4612 ms) 2023-10-06T13:56:25.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2023-10-06T13:56:25.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2023-10-06T13:56:25.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94040975808720 err -107 2023-10-06T13:56:25.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: watch_notify2_test_cb from 4494 notify_id 283467841538 cookie 94040975808720 2023-10-06T13:56:25.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (7301 ms) 2023-10-06T13:56:25.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2023-10-06T13:56:25.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2023-10-06T13:56:25.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94040975808720 err -107 2023-10-06T13:56:25.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1350 ms) 2023-10-06T13:56:25.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (64115 ms total) 2023-10-06T13:56:25.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: 2023-10-06T13:56:25.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2023-10-06T13:56:25.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2023-10-06T13:56:25.036 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: watch_notify_test_cb 2023-10-06T13:56:25.036 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (9411 ms) 2023-10-06T13:56:25.036 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (9411 ms total) 2023-10-06T13:56:25.036 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: 2023-10-06T13:56:25.036 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [----------] Global test environment tear-down 2023-10-06T13:56:25.036 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (130632 ms total) 2023-10-06T13:56:25.036 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify: [ PASSED ] 11 tests. 2023-10-06T13:56:35.112 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:04:38.234 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:35.123 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:04:38.246 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:35.132 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:04:38.255 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:35.137 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:04:38.231 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:35.189 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:04:38.312 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:35.203 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:04:38.300 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:35.263 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:04:38.385 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:35.289 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:04:38.412 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:35.297 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: Running main() from gmock_main.cc 2023-10-06T13:56:35.297 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2023-10-06T13:56:35.297 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [----------] Global test environment set-up. 2023-10-06T13:56:35.297 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2023-10-06T13:56:35.297 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2023-10-06T13:56:35.297 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2023-10-06T13:56:35.297 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2023-10-06T13:56:35.298 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: 2023-10-06T13:56:35.298 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2023-10-06T13:56:35.298 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2023-10-06T13:56:35.298 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (32093 ms) 2023-10-06T13:56:35.298 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2023-10-06T13:56:35.298 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (24368 ms) 2023-10-06T13:56:35.298 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2023-10-06T13:56:35.298 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (35512 ms) 2023-10-06T13:56:35.298 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2023-10-06T13:56:35.298 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (33960 ms) 2023-10-06T13:56:35.298 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2023-10-06T13:56:35.299 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (4353 ms) 2023-10-06T13:56:35.299 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2023-10-06T13:56:35.299 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (6410 ms) 2023-10-06T13:56:35.299 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2023-10-06T13:56:35.299 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (4143 ms) 2023-10-06T13:56:35.299 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (140839 ms total) 2023-10-06T13:56:35.299 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: 2023-10-06T13:56:35.299 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [----------] Global test environment tear-down 2023-10-06T13:56:35.299 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (140839 ms total) 2023-10-06T13:56:35.299 INFO:tasks.workunit.client.0.smithi019.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2023-10-06T13:56:35.300 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:04:38.397 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:56:48.150 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2023-10-06T13:56:48.150 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2023-10-06T13:56:48.150 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2023-10-06T13:56:48.151 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2023-10-06T13:56:48.151 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2023-10-06T13:56:48.151 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: notify 2023-10-06T13:56:48.151 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (37757 ms) 2023-10-06T13:56:48.151 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2023-10-06T13:56:48.151 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (5429 ms) 2023-10-06T13:56:48.151 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (43186 ms total) 2023-10-06T13:56:48.151 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: 2023-10-06T13:56:48.151 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2023-10-06T13:56:48.151 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2023-10-06T13:56:48.151 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: notify 2023-10-06T13:56:48.151 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (179 ms) 2023-10-06T13:56:48.151 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2023-10-06T13:56:48.152 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: notify 2023-10-06T13:56:48.152 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (38003 ms) 2023-10-06T13:56:48.152 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2023-10-06T13:56:48.152 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (6069 ms) 2023-10-06T13:56:48.152 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2023-10-06T13:56:48.152 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (2962 ms) 2023-10-06T13:56:48.152 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2023-10-06T13:56:48.152 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: handle_notify cookie 94825734862640 notify_id 408021893120 notifier_gid 8183 2023-10-06T13:56:48.152 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (2701 ms) 2023-10-06T13:56:48.152 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2023-10-06T13:56:48.152 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: handle_notify cookie 94825734862640 notify_id 416611827715 notifier_gid 8183 2023-10-06T13:56:48.152 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (2009 ms) 2023-10-06T13:56:48.152 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2023-10-06T13:56:48.153 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: handle_notify cookie 94825733905120 notify_id 425201762306 notifier_gid 8183 2023-10-06T13:56:48.153 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (2199 ms) 2023-10-06T13:56:48.153 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2023-10-06T13:56:48.153 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: handle_notify cookie 94825733905120 notify_id 429496729603 notifier_gid 8183 2023-10-06T13:56:48.153 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (903 ms) 2023-10-06T13:56:48.153 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2023-10-06T13:56:48.153 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: handle_notify cookie 94825733915456 notify_id 450971566081 notifier_gid 8183 2023-10-06T13:56:48.153 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (4703 ms) 2023-10-06T13:56:48.153 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2023-10-06T13:56:48.153 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: handle_notify cookie 94825733915456 notify_id 455266533378 notifier_gid 8183 2023-10-06T13:56:48.153 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (2010 ms) 2023-10-06T13:56:48.153 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2023-10-06T13:56:48.154 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: trying... 2023-10-06T13:56:48.154 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: handle_notify cookie 94825733915456 notify_id 463856467969 notifier_gid 8183 2023-10-06T13:56:48.154 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: timed out 2023-10-06T13:57:05.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: flushing 2023-10-06T13:57:05.730 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: flushed 2023-10-06T13:57:05.730 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (4003 ms) 2023-10-06T13:57:05.730 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2023-10-06T13:57:05.730 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: trying... 2023-10-06T13:57:05.730 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: handle_notify cookie 94825733915456 notify_id 476741369860 notifier_gid 8183 2023-10-06T13:57:05.730 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: timed out 2023-10-06T13:57:05.730 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: flushing 2023-10-06T13:57:05.731 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: flushed 2023-10-06T13:57:05.731 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3519 ms) 2023-10-06T13:57:05.731 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2023-10-06T13:57:05.731 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: List watches 2023-10-06T13:57:05.732 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: notify2 2023-10-06T13:57:05.732 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: handle_notify cookie 94825733915456 notify_id 489626271746 notifier_gid 8183 2023-10-06T13:57:05.732 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: notify2 done 2023-10-06T13:57:05.732 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: watch_check 2023-10-06T13:57:05.732 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: unwatch2 2023-10-06T13:57:05.732 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: flushing 2023-10-06T13:57:05.732 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: done 2023-10-06T13:57:05.732 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (4094 ms) 2023-10-06T13:57:05.732 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2023-10-06T13:57:05.733 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: List watches 2023-10-06T13:57:05.733 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: notify2 2023-10-06T13:57:05.733 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: handle_notify cookie 94825733915456 notify_id 515396075525 notifier_gid 8183 2023-10-06T13:57:05.733 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: notify2 done 2023-10-06T13:57:05.733 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: watch_check 2023-10-06T13:57:05.733 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: unwatch2 2023-10-06T13:57:05.733 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: flushing 2023-10-06T13:57:05.733 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: done 2023-10-06T13:57:05.733 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (5101 ms) 2023-10-06T13:57:05.733 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (78455 ms total) 2023-10-06T13:57:05.733 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: 2023-10-06T13:57:05.733 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2023-10-06T13:57:05.734 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (171078 ms total) 2023-10-06T13:57:05.734 INFO:tasks.workunit.client.0.smithi019.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2023-10-06T13:57:15.682 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:05:18.804 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:15.688 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:05:18.811 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:15.758 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:05:18.855 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:15.760 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:05:18.857 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:15.771 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:05:18.894 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:15.779 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:05:18.902 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:15.835 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:05:18.957 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:15.874 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:05:18.971 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:15.935 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:05:19.058 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:19.979 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: Running main() from gmock_main.cc 2023-10-06T13:57:19.979 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2023-10-06T13:57:19.979 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [----------] Global test environment set-up. 2023-10-06T13:57:19.980 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2023-10-06T13:57:19.980 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2023-10-06T13:57:19.980 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (26494 ms) 2023-10-06T13:57:19.980 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2023-10-06T13:57:19.980 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (6849 ms) 2023-10-06T13:57:19.980 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2023-10-06T13:57:19.980 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (17838 ms) 2023-10-06T13:57:19.980 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2023-10-06T13:57:19.980 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (9636 ms) 2023-10-06T13:57:19.980 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (60817 ms total) 2023-10-06T13:57:19.980 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: 2023-10-06T13:57:19.980 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2023-10-06T13:57:19.981 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2023-10-06T13:57:19.981 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (34373 ms) 2023-10-06T13:57:19.981 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2023-10-06T13:57:19.981 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (9092 ms) 2023-10-06T13:57:19.981 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2023-10-06T13:57:19.981 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (8332 ms) 2023-10-06T13:57:19.981 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (51797 ms total) 2023-10-06T13:57:19.981 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: 2023-10-06T13:57:19.981 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2023-10-06T13:57:19.981 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2023-10-06T13:57:19.981 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (5702 ms) 2023-10-06T13:57:19.981 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2023-10-06T13:57:19.982 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2015 ms) 2023-10-06T13:57:19.982 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2023-10-06T13:57:19.982 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (4156 ms) 2023-10-06T13:57:19.982 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2023-10-06T13:57:19.982 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (3443 ms) 2023-10-06T13:57:19.982 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (15316 ms total) 2023-10-06T13:57:19.982 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: 2023-10-06T13:57:19.982 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2023-10-06T13:57:19.982 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2023-10-06T13:57:19.982 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (8733 ms) 2023-10-06T13:57:19.982 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2023-10-06T13:57:19.983 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (7009 ms) 2023-10-06T13:57:19.983 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (15742 ms total) 2023-10-06T13:57:19.983 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: 2023-10-06T13:57:19.983 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [----------] Global test environment tear-down 2023-10-06T13:57:19.983 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (185657 ms total) 2023-10-06T13:57:19.983 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots: [ PASSED ] 13 tests. 2023-10-06T13:57:34.010 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2023-10-06T13:57:34.010 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [==========] Running 19 tests from 4 test suites. 2023-10-06T13:57:34.010 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2023-10-06T13:57:34.010 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2023-10-06T13:57:34.010 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: seed 92102 2023-10-06T13:57:34.011 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2023-10-06T13:57:34.011 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (19668 ms) 2023-10-06T13:57:34.011 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2023-10-06T13:57:34.011 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (4109 ms) 2023-10-06T13:57:34.011 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2023-10-06T13:57:34.011 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (6214 ms) 2023-10-06T13:57:34.011 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2023-10-06T13:57:34.011 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (4461 ms) 2023-10-06T13:57:34.012 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2023-10-06T13:57:34.012 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (23543 ms) 2023-10-06T13:57:34.012 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (57995 ms total) 2023-10-06T13:57:34.012 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: 2023-10-06T13:57:34.012 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2023-10-06T13:57:34.012 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2023-10-06T13:57:34.012 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (32909 ms) 2023-10-06T13:57:34.012 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2023-10-06T13:57:34.013 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (11683 ms) 2023-10-06T13:57:34.013 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2023-10-06T13:57:34.013 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (12237 ms) 2023-10-06T13:57:34.013 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2023-10-06T13:57:34.013 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4054 ms) 2023-10-06T13:57:34.013 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2023-10-06T13:57:34.013 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (4497 ms) 2023-10-06T13:57:34.014 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2023-10-06T13:57:34.014 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.2.0-596-g4af54c4a/rpm/el9/BUILD/ceph-18.2.0-596-g4af54c4a/src/test/librados/snapshots_cxx.cc:463: Skipped 2023-10-06T13:57:34.014 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: 2023-10-06T13:57:34.014 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2023-10-06T13:57:34.014 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2023-10-06T13:57:34.015 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: deleting snap 23 in pool LibRadosSnapshotsSelfManagedPP_smithi019-92102-7 2023-10-06T13:57:34.015 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: waiting for snaps to purge 2023-10-06T13:57:34.015 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17462 ms) 2023-10-06T13:57:34.015 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (82842 ms total) 2023-10-06T13:57:34.016 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: 2023-10-06T13:57:34.016 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2023-10-06T13:57:34.016 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2023-10-06T13:57:34.016 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (4686 ms) 2023-10-06T13:57:34.016 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2023-10-06T13:57:34.016 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (3162 ms) 2023-10-06T13:57:34.016 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2023-10-06T13:57:34.016 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2576 ms) 2023-10-06T13:57:34.017 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2023-10-06T13:57:34.017 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2030 ms) 2023-10-06T13:57:34.017 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (12454 ms total) 2023-10-06T13:57:34.017 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: 2023-10-06T13:57:34.017 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2023-10-06T13:57:35.747 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:05:38.844 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:35.809 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:05:38.845 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:35.809 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:05:38.880 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:35.810 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:05:38.882 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:35.820 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:05:38.917 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:35.837 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:05:38.960 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:35.845 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:05:38.968 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:35.865 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:05:38.988 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:35.961 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:05:39.037 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:52.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2023-10-06T13:57:52.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (5157 ms) 2023-10-06T13:57:52.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2023-10-06T13:57:52.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (5227 ms) 2023-10-06T13:57:52.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2023-10-06T13:57:52.476 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4625 ms) 2023-10-06T13:57:52.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (15009 ms total) 2023-10-06T13:57:52.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: 2023-10-06T13:57:52.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2023-10-06T13:57:52.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [==========] 19 tests from 4 test suites ran. (218121 ms total) 2023-10-06T13:57:52.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ PASSED ] 18 tests. 2023-10-06T13:57:52.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2023-10-06T13:57:52.477 INFO:tasks.workunit.client.0.smithi019.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2023-10-06T13:57:55.755 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:05:58.877 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:55.757 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:05:58.880 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:55.802 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:05:58.899 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:55.804 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:05:58.901 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:55.817 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:05:58.940 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:55.830 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:05:58.953 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:55.831 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:05:58.954 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:55.839 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:05:58.936 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:57:55.881 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:05:59.004 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:02.316 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T13:58:02.307+0000 1125f640 -1 osd.6 148 get_health_metrics reporting 1 slow ops, oldest is osd_repop(osd.3.0:194 77.14 e147/92 77:2a30fabb:test-rados-api-smithi019-91898-12::bar:2 v 147'5, mlcod=138'1) 2023-10-06T13:58:12.081 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: Running main() from gmock_main.cc 2023-10-06T13:58:12.082 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2023-10-06T13:58:12.082 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [----------] Global test environment set-up. 2023-10-06T13:58:12.082 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [----------] 26 tests from LibRadosAio 2023-10-06T13:58:12.082 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2023-10-06T13:58:12.082 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.TooBig (4753 ms) 2023-10-06T13:58:12.082 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2023-10-06T13:58:12.082 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (35637 ms) 2023-10-06T13:58:12.082 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2023-10-06T13:58:12.082 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (10485 ms) 2023-10-06T13:58:12.083 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2023-10-06T13:58:12.083 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (22160 ms) 2023-10-06T13:58:12.083 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2023-10-06T13:58:12.083 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (45175 ms) 2023-10-06T13:58:12.083 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2023-10-06T13:58:12.083 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (6936 ms) 2023-10-06T13:58:12.083 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2023-10-06T13:58:12.083 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (5421 ms) 2023-10-06T13:58:12.083 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2023-10-06T13:58:12.083 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (6430 ms) 2023-10-06T13:58:12.083 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2023-10-06T13:58:12.083 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (4115 ms) 2023-10-06T13:58:12.083 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2023-10-06T13:58:12.084 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (4375 ms) 2023-10-06T13:58:12.084 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2023-10-06T13:58:12.084 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (2598 ms) 2023-10-06T13:58:12.084 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2023-10-06T13:58:12.084 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (4064 ms) 2023-10-06T13:58:12.084 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2023-10-06T13:58:12.084 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (5219 ms) 2023-10-06T13:58:12.084 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2023-10-06T13:58:12.084 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (3442 ms) 2023-10-06T13:58:12.084 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2023-10-06T13:58:12.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.Flush (2709 ms) 2023-10-06T13:58:12.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2023-10-06T13:58:12.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3068 ms) 2023-10-06T13:58:12.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2023-10-06T13:58:12.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3182 ms) 2023-10-06T13:58:12.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2023-10-06T13:58:12.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3707 ms) 2023-10-06T13:58:12.085 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2023-10-06T13:58:12.086 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3299 ms) 2023-10-06T13:58:12.086 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2023-10-06T13:58:12.086 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (4335 ms) 2023-10-06T13:58:12.086 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2023-10-06T13:58:12.086 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (4692 ms) 2023-10-06T13:58:12.086 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2023-10-06T13:58:12.086 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (4273 ms) 2023-10-06T13:58:12.086 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2023-10-06T13:58:12.086 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3399 ms) 2023-10-06T13:58:12.086 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2023-10-06T13:58:12.086 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (4117 ms) 2023-10-06T13:58:12.086 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2023-10-06T13:58:12.087 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (36265 ms) 2023-10-06T13:58:12.087 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2023-10-06T13:58:12.087 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (4052 ms) 2023-10-06T13:58:12.087 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [----------] 26 tests from LibRadosAio (237908 ms total) 2023-10-06T13:58:12.087 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: 2023-10-06T13:58:12.087 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2023-10-06T13:58:16.157 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:06:19.280 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:16.161 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:06:19.284 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:16.213 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:06:19.310 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:16.219 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:06:19.316 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:16.235 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:06:19.358 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:16.236 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:06:19.359 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:16.239 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:06:19.362 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:16.259 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:06:19.356 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:16.301 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:06:19.423 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:19.290 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleW api_aio_pp: Running main() from gmock_main.cc 2023-10-06T13:58:19.290 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2023-10-06T13:58:19.290 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [----------] Global test environment set-up. 2023-10-06T13:58:19.291 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2023-10-06T13:58:19.291 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2023-10-06T13:58:19.291 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (4860 ms) 2023-10-06T13:58:19.291 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2023-10-06T13:58:19.291 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (41938 ms) 2023-10-06T13:58:19.291 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2023-10-06T13:58:19.291 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (53068 ms) 2023-10-06T13:58:19.291 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2023-10-06T13:58:19.291 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (25276 ms) 2023-10-06T13:58:19.291 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2023-10-06T13:58:19.291 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (4372 ms) 2023-10-06T13:58:19.292 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2023-10-06T13:58:19.292 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (5724 ms) 2023-10-06T13:58:19.292 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2023-10-06T13:58:19.292 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (5872 ms) 2023-10-06T13:58:19.292 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2023-10-06T13:58:19.292 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3260 ms) 2023-10-06T13:58:19.292 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2023-10-06T13:58:19.292 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (3626 ms) 2023-10-06T13:58:19.292 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2023-10-06T13:58:19.292 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (4136 ms) 2023-10-06T13:58:19.292 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2023-10-06T13:58:19.292 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (5214 ms) 2023-10-06T13:58:19.292 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2023-10-06T13:58:19.293 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3483 ms) 2023-10-06T13:58:19.293 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2023-10-06T13:58:19.293 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3684 ms) 2023-10-06T13:58:19.293 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2023-10-06T13:58:19.293 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3117 ms) 2023-10-06T13:58:19.293 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2023-10-06T13:58:19.293 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (5836 ms) 2023-10-06T13:58:19.293 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2023-10-06T13:58:19.293 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (4295 ms) 2023-10-06T13:58:19.293 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2023-10-06T13:58:19.293 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3269 ms) 2023-10-06T13:58:19.293 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2023-10-06T13:58:19.293 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (2569 ms) 2023-10-06T13:58:19.294 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2023-10-06T13:58:19.294 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (4347 ms) 2023-10-06T13:58:19.294 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2023-10-06T13:58:19.294 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3067 ms) 2023-10-06T13:58:19.294 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2023-10-06T13:58:19.294 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (2452 ms) 2023-10-06T13:58:19.294 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2023-10-06T13:58:19.294 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (4143 ms) 2023-10-06T13:58:19.294 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2023-10-06T13:58:19.295 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3213 ms) 2023-10-06T13:58:19.295 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2023-10-06T13:58:19.295 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (33036 ms) 2023-10-06T13:58:19.295 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2023-10-06T13:58:19.295 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3983 ms) 2023-10-06T13:58:19.295 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2023-10-06T13:58:19.295 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3922 ms) 2023-10-06T13:58:19.295 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2023-10-06T13:58:36.013 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:06:39.136 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:36.019 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:06:39.142 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:36.071 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:06:39.194 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:36.072 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:06:39.169 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:36.075 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:06:39.172 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:36.077 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:06:39.200 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:36.080 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:06:39.203 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:36.122 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:06:39.219 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:36.143 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:06:39.266 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:58:47.130 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRados api_misc_pp: [==========] Running 31 tests from 7 test suites. 2023-10-06T13:58:47.130 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] Global test environment set-up. 2023-10-06T13:58:47.131 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2023-10-06T13:58:47.131 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2023-10-06T13:58:47.131 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2023-10-06T13:58:47.131 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2023-10-06T13:58:47.131 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: 2023-10-06T13:58:47.131 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2023-10-06T13:58:47.131 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: seed 91864 2023-10-06T13:58:47.131 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2023-10-06T13:58:47.131 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (8 ms) 2023-10-06T13:58:47.131 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2023-10-06T13:58:47.131 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (44249 ms) 2023-10-06T13:58:47.131 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2023-10-06T13:58:47.132 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (4647 ms) 2023-10-06T13:58:47.132 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2023-10-06T13:58:47.132 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (432 ms) 2023-10-06T13:58:47.132 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2023-10-06T13:58:47.132 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (537 ms) 2023-10-06T13:58:47.132 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2023-10-06T13:58:47.132 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (92 ms) 2023-10-06T13:58:47.132 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2023-10-06T13:58:47.132 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (33 ms) 2023-10-06T13:58:47.132 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2023-10-06T13:58:47.132 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (155 ms) 2023-10-06T13:58:47.132 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2023-10-06T13:58:47.133 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (61 ms) 2023-10-06T13:58:47.133 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2023-10-06T13:58:47.133 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (194 ms) 2023-10-06T13:58:47.133 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2023-10-06T13:58:47.133 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (69 ms) 2023-10-06T13:58:47.133 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2023-10-06T13:58:47.133 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (89 ms) 2023-10-06T13:58:47.133 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2023-10-06T13:58:47.133 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (282 ms) 2023-10-06T13:58:47.133 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2023-10-06T13:58:47.133 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: osd_max_attr_size = 0 2023-10-06T13:58:47.133 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2023-10-06T13:58:47.133 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (96859 ms) 2023-10-06T13:58:47.134 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2023-10-06T13:58:47.134 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (12306 ms) 2023-10-06T13:58:47.134 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2023-10-06T13:58:47.134 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: waiting for initial deep scrubs... 2023-10-06T13:58:47.134 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: done waiting, doing copies 2023-10-06T13:58:47.134 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: waiting for final deep scrubs... 2023-10-06T13:58:47.134 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: done waiting 2023-10-06T13:58:47.134 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (102037 ms) 2023-10-06T13:58:47.134 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2023-10-06T13:58:47.134 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (129 ms) 2023-10-06T13:58:47.134 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2023-10-06T13:58:47.135 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (629 ms) 2023-10-06T13:58:47.135 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2023-10-06T13:58:47.135 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4507 ms) 2023-10-06T13:58:47.135 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2023-10-06T13:58:47.135 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2023-10-06T13:58:47.135 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2023-10-06T13:58:47.135 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2023-10-06T13:58:47.135 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2023-10-06T13:58:47.135 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2023-10-06T13:59:01.352 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:07:04.474 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:01.354 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:07:04.477 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:01.414 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:07:04.511 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:01.419 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:07:04.515 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:01.426 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:07:04.549 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:01.431 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:07:04.554 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:01.459 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:07:04.582 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:01.474 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:07:04.571 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:01.505 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:07:04.628 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:13.029 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] 22 tst: : entry=12 expected=12 2023-10-06T13:59:13.029 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:02547ec2:::1:head 2023-10-06T13:59:13.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:02547ec2:::1:head expected=10:02547ec2:::1:head 2023-10-06T13:59:13.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:02547ec2:::1:head -> 1 2023-10-06T13:59:13.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=1 expected=1 2023-10-06T13:59:13.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:bd63b0f1:::8:head 2023-10-06T13:59:13.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:bd63b0f1:::8:head expected=10:bd63b0f1:::8:head 2023-10-06T13:59:13.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:bd63b0f1:::8:head -> 8 2023-10-06T13:59:13.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=8 expected=8 2023-10-06T13:59:13.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:b29083e3:::5:head 2023-10-06T13:59:13.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:b29083e3:::5:head expected=10:b29083e3:::5:head 2023-10-06T13:59:13.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:b29083e3:::5:head -> 5 2023-10-06T13:59:13.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=5 expected=5 2023-10-06T13:59:13.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:89d3ae78:::11:head 2023-10-06T13:59:13.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:89d3ae78:::11:head expected=10:89d3ae78:::11:head 2023-10-06T13:59:13.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:89d3ae78:::11:head -> 11 2023-10-06T13:59:13.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=11 expected=11 2023-10-06T13:59:13.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:863748b0:::15:head 2023-10-06T13:59:13.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:863748b0:::15:head expected=10:863748b0:::15:head 2023-10-06T13:59:13.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:863748b0:::15:head -> 15 2023-10-06T13:59:13.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=15 expected=15 2023-10-06T13:59:13.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:6cac518f:::0:head 2023-10-06T13:59:13.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:6cac518f:::0:head expected=10:6cac518f:::0:head 2023-10-06T13:59:13.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:6cac518f:::0:head -> 0 2023-10-06T13:59:13.031 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=0 expected=0 2023-10-06T13:59:13.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:5c6b0b28:::7:head 2023-10-06T13:59:13.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:5c6b0b28:::7:head expected=10:5c6b0b28:::7:head 2023-10-06T13:59:13.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:5c6b0b28:::7:head -> 7 2023-10-06T13:59:13.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=7 expected=7 2023-10-06T13:59:13.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : seek to 10:566253c9:::13:head 2023-10-06T13:59:13.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : cursor()=10:566253c9:::13:head expected=10:566253c9:::13:head 2023-10-06T13:59:13.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: > 10:566253c9:::13:head -> 13 2023-10-06T13:59:13.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: : entry=13 expected=13 2023-10-06T13:59:13.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (7190 ms) 2023-10-06T13:59:13.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2023-10-06T13:59:13.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (184952 ms) 2023-10-06T13:59:13.032 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2023-10-06T13:59:13.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: split 0/5 -> MIN 10:33333333::::head 2023-10-06T13:59:13.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: split 1/5 -> 10:33333333::::head 10:66666666::::head 2023-10-06T13:59:13.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: split 2/5 -> 10:66666666::::head 10:99999999::::head 2023-10-06T13:59:13.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: split 3/5 -> 10:99999999::::head 10:cccccccc::::head 2023-10-06T13:59:13.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: split 4/5 -> 10:cccccccc::::head MAX 2023-10-06T13:59:13.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (20576 ms) 2023-10-06T13:59:13.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [----------] 7 tests from LibRadosList (283238 ms total) 2023-10-06T13:59:13.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 2023-10-06T13:59:13.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [----------] 3 tests from LibRadosListEC 2023-10-06T13:59:13.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2023-10-06T13:59:13.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1809 ms) 2023-10-06T13:59:13.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2023-10-06T13:59:13.033 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: myset foo1,foo2,foo3 2023-10-06T13:59:13.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo1 2023-10-06T13:59:13.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo2 2023-10-06T13:59:13.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo3 2023-10-06T13:59:13.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: myset foo1,foo4,foo5 2023-10-06T13:59:13.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo4 2023-10-06T13:59:13.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo5 2023-10-06T13:59:13.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo1 2023-10-06T13:59:13.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: myset foo6,foo7 2023-10-06T13:59:13.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo7 2023-10-06T13:59:13.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: foo6 2023-10-06T13:59:13.034 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2023-10-06T13:59:13.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: ns1:foo4 2023-10-06T13:59:13.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: ns1:foo5 2023-10-06T13:59:13.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: ns2:foo7 2023-10-06T13:59:13.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: ns2:foo6 2023-10-06T13:59:13.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: ns1:foo1 2023-10-06T13:59:13.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: :foo1 2023-10-06T13:59:13.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: :foo2 2023-10-06T13:59:13.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: :foo3 2023-10-06T13:59:13.035 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (4558 ms) 2023-10-06T13:59:21.349 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:07:24.471 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:21.401 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:07:24.524 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:21.481 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:07:24.604 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:21.524 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:07:24.620 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:21.525 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:07:24.622 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:21.532 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:07:24.655 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:21.534 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:07:24.657 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:21.578 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:07:24.674 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:21.583 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:07:24.706 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:24.604 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2023-10-06T13:59:24.605 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 1 0 2023-10-06T13:59:24.605 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 10 0 2023-10-06T13:59:24.605 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 13 0 2023-10-06T13:59:24.605 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 7 0 2023-10-06T13:59:24.605 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 14 0 2023-10-06T13:59:24.605 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 0 0 2023-10-06T13:59:24.605 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 15 0 2023-10-06T13:59:24.605 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 11 0 2023-10-06T13:59:24.605 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 5 0 2023-10-06T13:59:24.605 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 8 0 2023-10-06T13:59:24.606 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 6 0 2023-10-06T13:59:24.606 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 3 0 2023-10-06T13:59:24.606 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 4 0 2023-10-06T13:59:24.606 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 12 0 2023-10-06T13:59:24.606 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 9 0 2023-10-06T13:59:24.606 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 2 0 2023-10-06T13:59:24.606 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2023-10-06T13:59:24.606 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (4854 ms) 2023-10-06T13:59:24.606 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [----------] 3 tests from LibRadosListEC (11222 ms total) 2023-10-06T13:59:24.606 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 2023-10-06T13:59:24.606 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [----------] 1 test from LibRadosListNP 2023-10-06T13:59:24.607 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2023-10-06T13:59:24.607 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3293 ms) 2023-10-06T13:59:24.607 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [----------] 1 test from LibRadosListNP (3293 ms total) 2023-10-06T13:59:24.607 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: 2023-10-06T13:59:24.607 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [----------] Global test environment tear-down 2023-10-06T13:59:24.607 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [==========] 11 tests from 3 test suites ran. (310371 ms total) 2023-10-06T13:59:24.607 INFO:tasks.workunit.client.0.smithi019.stdout: api_list: [ PASSED ] 11 tests. 2023-10-06T13:59:30.778 INFO:tasks.workunit.client.0.smithi019.stdout:ests from LibRadosMiscPP (267315 ms total) 2023-10-06T13:59:30.779 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: 2023-10-06T13:59:30.779 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2023-10-06T13:59:30.779 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2023-10-06T13:59:30.779 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (11370 ms) 2023-10-06T13:59:30.779 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (11370 ms total) 2023-10-06T13:59:30.779 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: 2023-10-06T13:59:30.779 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2023-10-06T13:59:30.779 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2023-10-06T13:59:30.779 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (634 ms) 2023-10-06T13:59:30.779 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2023-10-06T13:59:30.779 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (1220 ms) 2023-10-06T13:59:30.779 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (1854 ms total) 2023-10-06T13:59:30.780 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: 2023-10-06T13:59:30.780 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2023-10-06T13:59:30.780 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2023-10-06T13:59:30.780 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (360 ms) 2023-10-06T13:59:30.780 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2023-10-06T13:59:30.780 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (492 ms) 2023-10-06T13:59:30.780 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (852 ms total) 2023-10-06T13:59:30.780 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: 2023-10-06T13:59:30.780 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2023-10-06T13:59:30.780 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2023-10-06T13:59:30.780 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (932 ms) 2023-10-06T13:59:30.780 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2023-10-06T13:59:30.781 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (2106 ms) 2023-10-06T13:59:30.781 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (3038 ms total) 2023-10-06T13:59:30.781 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: 2023-10-06T13:59:30.781 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2023-10-06T13:59:30.781 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2023-10-06T13:59:30.781 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1473 ms) 2023-10-06T13:59:30.781 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1473 ms total) 2023-10-06T13:59:30.781 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: 2023-10-06T13:59:30.781 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [----------] Global test environment tear-down 2023-10-06T13:59:30.781 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (316498 ms total) 2023-10-06T13:59:30.781 INFO:tasks.workunit.client.0.smithi019.stdout: api_misc_pp: [ PASSED ] 31 tests. 2023-10-06T13:59:33.157 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:07:36.279 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:33.161 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:07:36.284 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:33.216 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:07:36.339 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:33.217 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:07:36.313 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:33.219 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:07:36.316 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:33.242 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:07:36.364 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:33.244 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:07:36.366 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:33.254 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:07:36.351 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:33.293 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:07:36.414 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:41.424 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:07:44.546 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:41.425 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:07:44.548 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:41.461 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:07:44.584 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:41.467 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:07:44.564 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:41.473 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:07:44.570 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:41.492 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:07:44.615 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:41.498 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:07:44.594 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:41.498 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:07:44.619 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T13:59:41.520 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:07:44.643 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:01.691 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:08:04.813 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:01.693 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:08:04.816 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:01.737 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:08:04.859 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:01.743 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:08:04.840 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:01.747 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:08:04.844 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:01.749 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:08:04.872 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:01.753 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:08:04.876 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:01.787 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:08:04.884 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:01.804 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:08:04.927 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:17.027 INFO:tasks.workunit.client.0.smithi019.stdout:rite 2023-10-06T14:00:17.027 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (8372 ms) 2023-10-06T14:00:17.027 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2023-10-06T14:00:17.027 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (7919 ms) 2023-10-06T14:00:17.028 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2023-10-06T14:00:17.028 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (9687 ms) 2023-10-06T14:00:17.028 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2023-10-06T14:00:17.028 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7801 ms) 2023-10-06T14:00:17.028 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2023-10-06T14:00:17.028 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (8118 ms) 2023-10-06T14:00:17.028 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2023-10-06T14:00:17.028 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7454 ms) 2023-10-06T14:00:17.028 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2023-10-06T14:00:17.028 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7291 ms) 2023-10-06T14:00:17.028 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2023-10-06T14:00:17.028 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7125 ms) 2023-10-06T14:00:17.029 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2023-10-06T14:00:17.029 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (10558 ms) 2023-10-06T14:00:17.029 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2023-10-06T14:00:17.029 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7672 ms) 2023-10-06T14:00:17.029 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2023-10-06T14:00:17.029 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6813 ms) 2023-10-06T14:00:17.029 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2023-10-06T14:00:17.029 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (7982 ms) 2023-10-06T14:00:17.029 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2023-10-06T14:00:17.029 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6762 ms) 2023-10-06T14:00:17.029 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2023-10-06T14:00:17.029 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7115 ms) 2023-10-06T14:00:17.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2023-10-06T14:00:17.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7489 ms) 2023-10-06T14:00:17.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2023-10-06T14:00:17.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (6786 ms) 2023-10-06T14:00:17.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (124944 ms total) 2023-10-06T14:00:17.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: 2023-10-06T14:00:17.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [----------] Global test environment tear-down 2023-10-06T14:00:17.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (362852 ms total) 2023-10-06T14:00:17.030 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio: [ PASSED ] 42 tests. 2023-10-06T14:00:26.900 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:08:30.022 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:26.903 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:08:30.026 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:26.955 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:08:30.078 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:26.957 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:08:30.054 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:26.958 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:08:30.055 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:26.961 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:08:30.084 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:27.012 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:08:30.109 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:27.030 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:08:30.153 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:27.115 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:08:30.238 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:46.966 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:08:50.089 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:46.970 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:08:50.093 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:47.015 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:08:50.112 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:47.022 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:08:50.145 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:47.025 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:08:50.122 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:47.040 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:08:50.163 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:47.042 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:08:50.165 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:47.063 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:08:50.160 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:00:47.104 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:08:50.227 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:06.893 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:09:10.015 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:06.912 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:09:10.021 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:06.943 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:09:10.040 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:06.948 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:09:10.071 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:06.979 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:09:10.046 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:06.979 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:09:10.081 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:06.979 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:09:10.092 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:06.980 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:09:10.076 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:07.000 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:09:10.123 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:24.673 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:24.666+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:25.077 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:25.075+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:25.694 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:25.691+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:26.047 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:26.045+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:26.655 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:26.653+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:26.939 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:09:30.062 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:26.942 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:09:30.065 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:26.961 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:09:30.084 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:26.988 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:09:30.084 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:26.990 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:09:30.087 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:26.999 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:09:30.122 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:27.001 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:09:30.124 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:27.008 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:09:30.104 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:27.019 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:09:30.141 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:27.033 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:27.031+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:27.656 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:27.652+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:28.050 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:28.048+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:28.613 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:28.609+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:29.096 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:29.094+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:29.636 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:29.634+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:30.129 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:30.127+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:30.656 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:30.654+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:31.172 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:31.169+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:31.695 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:31.692+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:32.203 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:32.200+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:32.705 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:32.702+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:33.191 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:33.189+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:33.739 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:33.736+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:34.213 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:34.211+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:34.702 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:34.700+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:35.194 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:35.192+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:35.673 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:35.670+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:35.720 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:09:38.843 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:35.723 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:09:38.846 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:35.732 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:09:38.829 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:35.733 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:09:38.830 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:35.734 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:09:38.857 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:35.782 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:09:38.905 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:35.792 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:09:38.889 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:35.802 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:09:38.925 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:35.810 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:09:38.933 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:36.229 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:36.225+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:36.664 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:36.661+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:37.183 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:37.180+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:37.618 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:37.616+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:38.191 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:38.188+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:38.635 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:38.632+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:39.150 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:39.147+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:39.611 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:39.608+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:40.117 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:40.113+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:40.582 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:40.579+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181.addd4234 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:41.107 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:01:41.104+0000 1125f640 -1 osd.6 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20671.0:1 182.4s0 182.4dc1d21c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e337) 2023-10-06T14:01:41.547 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:01:41.544+0000 10a5e640 -1 osd.2 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:1635 181.14 181:2c42bbb5:test-rados-api-smithi019-91898-20::foo:head [writefull 0~3 in=3b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336) 2023-10-06T14:01:44.206 INFO:tasks.workunit.client.0.smithi019.stdout:Aio.RoundTripAppendPP (3347 ms) 2023-10-06T14:01:44.206 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2023-10-06T14:01:44.206 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (4432 ms) 2023-10-06T14:01:44.206 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2023-10-06T14:01:44.206 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3607 ms) 2023-10-06T14:01:44.207 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2023-10-06T14:01:44.207 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3334 ms) 2023-10-06T14:01:44.207 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2023-10-06T14:01:44.207 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: setting pool EIO 2023-10-06T14:01:44.207 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: max_success 0, min_failed 1 2023-10-06T14:01:44.207 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (6308 ms) 2023-10-06T14:01:44.207 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2023-10-06T14:01:44.207 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (4211 ms) 2023-10-06T14:01:44.207 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (267004 ms total) 2023-10-06T14:01:44.207 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: 2023-10-06T14:01:44.207 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2023-10-06T14:01:44.207 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2023-10-06T14:01:44.208 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3503 ms) 2023-10-06T14:01:44.208 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2023-10-06T14:01:44.208 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (11558 ms) 2023-10-06T14:01:44.208 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2023-10-06T14:01:44.208 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (17101 ms) 2023-10-06T14:01:44.208 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2023-10-06T14:01:44.208 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3462 ms) 2023-10-06T14:01:44.208 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (35624 ms total) 2023-10-06T14:01:44.208 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: 2023-10-06T14:01:44.208 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2023-10-06T14:01:44.208 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2023-10-06T14:01:44.208 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3433 ms) 2023-10-06T14:01:44.209 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3433 ms total) 2023-10-06T14:01:44.209 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: 2023-10-06T14:01:44.209 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2023-10-06T14:01:44.209 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2023-10-06T14:01:44.209 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (14825 ms) 2023-10-06T14:01:44.209 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2023-10-06T14:01:44.209 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (7113 ms) 2023-10-06T14:01:44.209 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2023-10-06T14:01:44.209 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7705 ms) 2023-10-06T14:01:44.209 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2023-10-06T14:01:44.209 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (6903 ms) 2023-10-06T14:01:44.209 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2023-10-06T14:01:44.210 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3045 ms) 2023-10-06T14:01:44.210 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2023-10-06T14:01:44.210 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7199 ms) 2023-10-06T14:01:44.210 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2023-10-06T14:01:44.210 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (7750 ms) 2023-10-06T14:01:44.210 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2023-10-06T14:01:44.210 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7033 ms) 2023-10-06T14:01:44.210 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2023-10-06T14:01:44.210 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (8517 ms) 2023-10-06T14:01:44.210 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2023-10-06T14:01:44.210 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (6913 ms) 2023-10-06T14:01:44.210 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2023-10-06T14:01:44.211 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7939 ms) 2023-10-06T14:01:44.211 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2023-10-06T14:01:44.211 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (6808 ms) 2023-10-06T14:01:44.211 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2023-10-06T14:01:44.211 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (52216 ms) 2023-10-06T14:01:47.017 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:09:50.139 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:47.023 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:09:50.146 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:47.048 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:09:50.171 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:47.062 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:09:50.159 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:47.064 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:09:50.161 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:47.067 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:09:50.190 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:47.069 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:09:50.192 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:47.087 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:09:50.184 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:01:47.093 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:09:50.216 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:07.279 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:10:10.402 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:07.285 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:10:10.408 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:07.339 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:10:10.462 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:07.339 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:10:10.436 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:07.341 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:10:10.438 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:07.344 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:10:10.467 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:07.349 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:10:10.472 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:07.386 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:10:10.509 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:07.391 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:10:10.488 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:27.189 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:10:30.312 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:27.191 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:10:30.314 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:27.231 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:10:30.354 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:27.233 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:10:30.330 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:27.235 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:10:30.332 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:27.257 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:10:30.380 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:27.258 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:10:30.381 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:27.260 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:10:30.357 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:27.283 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:10:30.406 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:47.235 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:10:50.357 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:47.238 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:10:50.361 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:47.257 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:10:50.379 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:47.288 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:10:50.384 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:47.290 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:10:50.387 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:47.308 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:10:50.431 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:47.310 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:10:50.432 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:47.310 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:10:50.407 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:47.334 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:10:50.457 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:02:56.477 INFO:tasks.ceph.osd.3.smithi019.stderr:2023-10-06T14:02:20.346+0000 13263640 -1 bdev(0x1715e7f0 /var/lib/ceph/osd/ceph-3/block) debug_aio_unlink stalled aio 0x60c3140 since 2023-10-06T14:02:07.784793+0000, timeout is 5s 2023-10-06T14:03:07.390 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:11:10.512 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:07.394 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:11:10.517 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:07.421 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:11:10.544 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:07.448 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:11:10.543 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:07.448 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:11:10.544 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:07.455 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:11:10.578 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:07.457 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:11:10.580 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:07.472 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:11:10.569 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:07.489 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:11:10.611 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:16.790 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2023-10-06T14:03:16.790 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3272 ms) 2023-10-06T14:03:16.790 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2023-10-06T14:03:16.790 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7925 ms) 2023-10-06T14:03:16.790 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2023-10-06T14:03:16.790 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (6979 ms) 2023-10-06T14:03:16.790 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2023-10-06T14:03:16.790 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (8123 ms) 2023-10-06T14:03:16.791 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2023-10-06T14:03:16.791 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (51490 ms) 2023-10-06T14:03:16.791 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2023-10-06T14:03:16.791 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7409 ms) 2023-10-06T14:03:16.791 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2023-10-06T14:03:16.791 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7382 ms) 2023-10-06T14:03:16.791 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (236547 ms total) 2023-10-06T14:03:16.791 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: 2023-10-06T14:03:16.791 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [----------] Global test environment tear-down 2023-10-06T14:03:16.791 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (542608 ms total) 2023-10-06T14:03:16.791 INFO:tasks.workunit.client.0.smithi019.stdout: api_aio_pp: [ PASSED ] 57 tests. 2023-10-06T14:03:27.527 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:11:30.649 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:27.539 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:11:30.652 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:27.565 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:11:30.688 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:27.598 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:11:30.671 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:27.598 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:11:30.673 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:27.599 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:11:30.704 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:27.599 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:11:30.705 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:27.621 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:11:30.744 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:27.695 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:11:30.724 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:47.450 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:11:50.573 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:47.477 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:11:50.600 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:47.502 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:11:50.599 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:47.504 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:11:50.601 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:47.514 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:11:50.637 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:47.526 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:11:50.623 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:47.541 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:11:50.664 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:47.543 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:11:50.666 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:47.576 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:11:50.699 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:52.771 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:11:55.893 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:52.774 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:11:55.897 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:52.789 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:11:55.912 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:52.830 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:11:55.953 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:52.831 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:11:55.954 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:52.832 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:11:55.929 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:52.833 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:11:55.930 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:52.846 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:11:55.969 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:03:52.847 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:11:55.944 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:07.487 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:10.609 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:07.491 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:10.614 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:07.532 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:10.655 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:07.536 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:10.633 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:07.539 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:10.636 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:07.542 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:10.665 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:07.545 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:10.668 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:07.557 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:10.654 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:07.565 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:10.688 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:27.565 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:30.688 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:27.568 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:30.691 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:27.593 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:30.715 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:27.611 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:30.708 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:27.612 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:30.709 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:27.625 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:30.748 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:27.629 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:30.752 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:27.630 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:30.727 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:27.646 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:30.769 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:33.020 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:33.012+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:34.003 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:34.001+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:34.969 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:34.966+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:35.956 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:35.953+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:36.910 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:36.907+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:37.602 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:40.725 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:37.604 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:40.727 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:37.619 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:40.742 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:37.648 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:40.745 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:37.650 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:40.747 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:37.656 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:40.779 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:37.657 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:40.780 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:37.662 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:40.759 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:37.670 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:40.793 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:37.950 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:37.948+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:38.946 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:38.944+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:39.926 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:39.923+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:40.920 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:40.917+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:41.905 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:41.902+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:42.933 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:42.930+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:43.926 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:43.923+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:44.960 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:44.956+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:45.921 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:45.919+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:46.487 INFO:tasks.ceph.osd.5.smithi083.stderr:2023-10-06T14:04:46.481+0000 10a5e640 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:23.411382+0000 front 2023-10-06T14:03:23.410767+0000 (oldest deadline 2023-10-06T14:04:45.697025+0000) 2023-10-06T14:04:46.804 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:04:46.798+0000 10a5e640 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:23.599763+0000 front 2023-10-06T14:03:23.596985+0000 (oldest deadline 2023-10-06T14:04:45.960873+0000) 2023-10-06T14:04:46.886 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:46.884+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:47.496 INFO:tasks.ceph.osd.5.smithi083.stderr:2023-10-06T14:04:47.490+0000 10a5e640 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:23.411382+0000 front 2023-10-06T14:03:23.410767+0000 (oldest deadline 2023-10-06T14:04:45.697025+0000) 2023-10-06T14:04:47.655 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:50.777 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:47.657 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:50.780 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:47.672 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:50.795 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:47.705 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:50.802 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:47.706 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:50.828 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:47.707 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:50.804 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:47.709 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:50.832 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:47.719 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:50.816 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:47.724 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:50.847 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:47.787 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:04:47.785+0000 10a5e640 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:23.599763+0000 front 2023-10-06T14:03:23.596985+0000 (oldest deadline 2023-10-06T14:04:45.960873+0000) 2023-10-06T14:04:47.873 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:47.869+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:48.056 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:04:48.050+0000 10a5e640 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:24.348625+0000 front 2023-10-06T14:03:24.347784+0000 (oldest deadline 2023-10-06T14:04:47.834768+0000) 2023-10-06T14:04:48.224 INFO:tasks.ceph.osd.3.smithi019.stderr:2023-10-06T14:04:48.218+0000 10a5e640 -1 osd.3 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:23.298260+0000 front 2023-10-06T14:03:23.303470+0000 (oldest deadline 2023-10-06T14:04:47.992123+0000) 2023-10-06T14:04:48.476 INFO:tasks.ceph.osd.5.smithi083.stderr:2023-10-06T14:04:48.474+0000 10a5e640 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:23.411382+0000 front 2023-10-06T14:03:23.410767+0000 (oldest deadline 2023-10-06T14:04:45.697025+0000) 2023-10-06T14:04:48.563 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:04:48.557+0000 1125f640 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:25.258181+0000 front 2023-10-06T14:03:25.265545+0000 (oldest deadline 2023-10-06T14:04:48.140800+0000) 2023-10-06T14:04:48.783 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:04:48.780+0000 10a5e640 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:23.599763+0000 front 2023-10-06T14:03:23.596985+0000 (oldest deadline 2023-10-06T14:04:45.960873+0000) 2023-10-06T14:04:48.902 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:48.899+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:49.068 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:04:49.064+0000 10a5e640 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:24.348625+0000 front 2023-10-06T14:03:24.347784+0000 (oldest deadline 2023-10-06T14:04:47.834768+0000) 2023-10-06T14:04:49.186 INFO:tasks.ceph.osd.3.smithi019.stderr:2023-10-06T14:04:49.183+0000 10a5e640 -1 osd.3 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:23.298260+0000 front 2023-10-06T14:03:23.303470+0000 (oldest deadline 2023-10-06T14:04:47.992123+0000) 2023-10-06T14:04:49.447 INFO:tasks.ceph.osd.5.smithi083.stderr:2023-10-06T14:04:49.444+0000 10a5e640 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:23.411382+0000 front 2023-10-06T14:03:23.410767+0000 (oldest deadline 2023-10-06T14:04:45.697025+0000) 2023-10-06T14:04:49.612 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:04:49.605+0000 1125f640 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:25.258181+0000 front 2023-10-06T14:03:25.265545+0000 (oldest deadline 2023-10-06T14:04:48.140800+0000) 2023-10-06T14:04:49.750 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:04:49.743+0000 10a5e640 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:23.599763+0000 front 2023-10-06T14:03:23.596985+0000 (oldest deadline 2023-10-06T14:04:45.960873+0000) 2023-10-06T14:04:49.891 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:49.884+0000 10a5e640 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:23.256706+0000 front 2023-10-06T14:03:23.260945+0000 (oldest deadline 2023-10-06T14:04:49.149999+0000) 2023-10-06T14:04:49.891 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:49.888+0000 10a5e640 -1 osd.0 410 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:50.027 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:04:50.023+0000 10a5e640 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:24.348625+0000 front 2023-10-06T14:03:24.347784+0000 (oldest deadline 2023-10-06T14:04:47.834768+0000) 2023-10-06T14:04:50.215 INFO:tasks.ceph.osd.3.smithi019.stderr:2023-10-06T14:04:50.212+0000 10a5e640 -1 osd.3 410 heartbeat_check: no reply from 172.21.15.83:6830 osd.4 since back 2023-10-06T14:03:23.298260+0000 front 2023-10-06T14:03:23.303470+0000 (oldest deadline 2023-10-06T14:04:47.992123+0000) 2023-10-06T14:04:50.903 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:04:50.900+0000 10a5e640 -1 osd.0 411 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2387 194.4 194.4 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e410) 2023-10-06T14:04:52.774 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:55.897 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:52.778 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:55.901 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:52.791 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:12:55.914 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:52.826 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:55.949 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:52.827 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:55.950 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:52.828 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:55.925 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:52.829 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:55.926 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:52.843 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:12:55.940 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:04:52.846 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:12:55.969 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:07.713 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:10.836 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:07.717 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:10.840 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:07.747 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:10.870 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:07.769 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:10.866 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:07.772 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:10.869 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:07.791 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:10.914 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:07.796 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:10.919 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:07.806 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:10.903 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:07.827 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:10.950 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:08.562 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:11.659 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:08.563 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:11.660 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:08.573 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:11.670 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:08.666 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:11.789 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:08.667 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:11.790 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:08.681 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:11.803 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:08.723 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:11.846 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:08.724 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:11.847 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:08.735 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:11.857 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:10.746 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:05:10.742+0000 12261640 -1 osd.4 428 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T14:05:22.755 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:25.851 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:22.755 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:25.878 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:22.757 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:25.880 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:22.757 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:25.854 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:22.781 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:25.904 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:22.782 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:25.879 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:22.822 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:25.945 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:22.825 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:25.948 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:22.848 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:25.971 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:27.770 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:30.893 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:27.771 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:30.894 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:27.782 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:30.905 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:27.814 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:30.910 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:27.821 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:30.943 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:27.822 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:30.919 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:27.833 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:30.930 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:27.837 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:30.960 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:27.853 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:30.976 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:41.327 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:41.321+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:42.331 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:42.328+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:42.810 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:45.933 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:42.814 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:45.937 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:42.816 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:45.913 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:42.818 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:45.915 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:42.836 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:45.933 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:42.845 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:45.966 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:42.862 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:45.985 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:42.865 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:45.988 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:42.881 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:46.004 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:43.329 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:43.326+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:44.331 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:44.328+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:45.309 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:45.306+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:46.334 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:46.331+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:47.292 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:47.290+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:48.286 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:48.283+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:49.289 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:49.286+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:50.293 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:50.289+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:51.312 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:51.309+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:52.283 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:52.280+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:52.842 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:55.964 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:52.843 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:55.966 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:52.857 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:13:55.979 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:52.890 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:56.012 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:52.893 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:56.016 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:52.893 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:55.990 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:52.895 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:55.992 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:52.908 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:13:56.005 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:52.908 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:13:56.031 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:53.253 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:53.250+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:54.296 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:54.292+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:55.281 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:55.277+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:56.275 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:56.272+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:57.310 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:57.307+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:57.852 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:00.975 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:57.855 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:00.978 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:57.868 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:00.991 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:57.906 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:01.028 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:57.906 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:01.003 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:57.908 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:01.031 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:57.908 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:01.005 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:57.919 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:01.042 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:57.920 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:01.016 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:05:58.332 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:58.329+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:05:59.337 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:05:59.334+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:00.369 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:00.366+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:01.353 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:01.349+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:02.332 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:02.330+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:03.337 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:03.334+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:04.333 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:04.330+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:05.309 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:05.305+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:06.307 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:06.305+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:07.330 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:07.327+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:08.293 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:08.291+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:09.330 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:09.324+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:10.305 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:10.303+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:11.280 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:11.278+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:12.283 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:12.281+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:12.913 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:16.036 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:12.916 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:16.038 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:12.935 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:16.058 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:12.964 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:16.060 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:12.966 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:16.062 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:12.969 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:16.092 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:12.970 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:16.093 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:12.983 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:16.079 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:12.989 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:16.112 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:13.318 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:13.316+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:13.559 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:16.656 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:13.560 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:16.657 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:13.568 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:16.665 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:13.746 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:16.868 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:13.747 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:16.870 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:13.757 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:16.879 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:13.810 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:16.932 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:13.811 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:16.933 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:13.819 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:16.941 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:14.325 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:14.323+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:15.325 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:15.323+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:16.301 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:16.299+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:17.348 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:17.346+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:18.342 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:18.340+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:19.383 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:19.381+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:20.338 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:20.336+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:21.351 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:21.349+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:22.350 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:22.348+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:23.321 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:23.318+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:24.317 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:24.315+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:25.290 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:25.288+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:26.254 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:26.252+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:27.188 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:06:27.154+0000 13a64640 -1 bdev(0x179537f0 /var/lib/ceph/osd/ceph-1/block) debug_aio_unlink stalled aio 0x2f700dc0 since 2023-10-06T14:06:13.248550+0000, timeout is 5s 2023-10-06T14:06:27.297 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:27.295+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196.496ff58f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:28.268 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:06:28.265+0000 10a5e640 -1 osd.7 432 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:2717 196.f 196:f1aff692:test-rados-api-smithi019-91898-34::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e429) 2023-10-06T14:06:33.165 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:36.288 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:33.168 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:36.291 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:33.197 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:36.320 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:33.215 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:36.312 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:33.302 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:36.315 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:33.302 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:36.341 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:33.302 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:36.344 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:33.302 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:36.347 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:33.303 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:36.389 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:48.071 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:51.194 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:48.130 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:51.223 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:48.130 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:51.225 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:48.130 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:51.197 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:48.130 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:51.214 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:48.139 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:51.262 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:48.145 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:51.268 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:48.146 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:51.242 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:48.170 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:51.293 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:53.084 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:56.207 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:53.196 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:56.212 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:53.196 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:14:56.228 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:53.196 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:56.234 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:53.196 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:56.237 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:53.196 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:14:56.249 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:53.197 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:56.266 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:53.197 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:56.270 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:06:53.197 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:14:56.281 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:08.157 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:11.277 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:08.159 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:11.282 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:08.184 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:11.307 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:08.214 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:11.336 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:08.216 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:11.339 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:08.218 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:11.315 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:08.219 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:11.317 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:08.238 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:11.361 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:08.240 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:11.337 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:13.162 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:16.284 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:13.163 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:16.286 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:13.179 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:16.302 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:13.205 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:16.302 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:13.208 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:16.305 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:13.219 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:16.341 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:13.220 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:16.343 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:13.222 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:16.319 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:13.233 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:16.356 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:23.186 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:26.308 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:23.190 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:26.313 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:23.214 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:26.337 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:23.234 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:26.331 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:23.235 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:26.332 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:23.258 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:26.355 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:23.258 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:26.381 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:23.260 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:26.383 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:23.283 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:26.406 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:33.449 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:36.572 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:33.451 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:36.574 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:33.478 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:36.601 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:33.497 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:36.594 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:33.499 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:36.596 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:33.512 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:36.635 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:33.514 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:36.637 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:33.534 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:36.631 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:33.543 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:36.666 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:38.321 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:41.443 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:38.325 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:41.448 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:38.340 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:41.463 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:38.370 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:41.466 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:38.386 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:41.483 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:38.393 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:41.516 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:38.394 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:41.517 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:38.400 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:41.497 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:38.409 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:41.531 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:41.954 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2023-10-06T14:07:41.954 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [----------] Global test environment set-up. 2023-10-06T14:07:41.954 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2023-10-06T14:07:41.954 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: seed 91898 2023-10-06T14:07:41.954 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2023-10-06T14:07:41.955 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (47639 ms) 2023-10-06T14:07:41.955 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2023-10-06T14:07:41.955 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (80748 ms) 2023-10-06T14:07:41.955 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2023-10-06T14:07:41.955 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (707 ms) 2023-10-06T14:07:41.955 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (129094 ms total) 2023-10-06T14:07:41.955 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: 2023-10-06T14:07:41.955 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2023-10-06T14:07:41.955 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2023-10-06T14:07:41.956 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (19323 ms) 2023-10-06T14:07:41.956 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2023-10-06T14:07:41.956 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (23614 ms) 2023-10-06T14:07:41.956 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2023-10-06T14:07:41.956 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (60013 ms) 2023-10-06T14:07:41.956 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2023-10-06T14:07:41.956 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: my_snaps [4] 2023-10-06T14:07:41.956 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: my_snaps [5,4] 2023-10-06T14:07:41.956 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: my_snaps [6,5,4] 2023-10-06T14:07:41.956 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: my_snaps [7,6,5,4] 2023-10-06T14:07:41.957 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: promoting some heads 2023-10-06T14:07:41.957 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: promoting from clones for snap 7 2023-10-06T14:07:41.957 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: promoting from clones for snap 6 2023-10-06T14:07:41.957 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: promoting from clones for snap 5 2023-10-06T14:07:41.957 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: promoting from clones for snap 4 2023-10-06T14:07:41.957 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: waiting for scrubs... 2023-10-06T14:07:41.957 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: done waiting 2023-10-06T14:07:41.957 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (134751 ms) 2023-10-06T14:07:41.957 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2023-10-06T14:07:41.957 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (11529 ms) 2023-10-06T14:07:41.958 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2023-10-06T14:07:41.958 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (9488 ms) 2023-10-06T14:07:41.958 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2023-10-06T14:07:41.958 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (56201 ms) 2023-10-06T14:07:41.958 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2023-10-06T14:07:41.958 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (10231 ms) 2023-10-06T14:07:41.958 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2023-10-06T14:07:41.958 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (67594 ms) 2023-10-06T14:07:41.958 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2023-10-06T14:07:41.959 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (10502 ms) 2023-10-06T14:07:41.959 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2023-10-06T14:07:41.959 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (11002 ms) 2023-10-06T14:07:41.959 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2023-10-06T14:07:41.959 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (91013 ms) 2023-10-06T14:07:41.959 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2023-10-06T14:07:41.959 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (9650 ms) 2023-10-06T14:07:41.959 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2023-10-06T14:07:41.959 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (119596 ms) 2023-10-06T14:07:41.959 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2023-10-06T14:07:41.959 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (15392 ms) 2023-10-06T14:07:41.960 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2023-10-06T14:07:41.960 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (10327 ms) 2023-10-06T14:07:41.960 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2023-10-06T14:07:41.960 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (11479 ms) 2023-10-06T14:07:53.481 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:56.578 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:53.482 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:56.579 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:53.485 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:56.608 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:53.491 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:56.614 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:53.520 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:15:56.617 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:53.562 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:56.685 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:53.564 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:56.687 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:53.581 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:15:56.704 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:53.600 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:15:56.723 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:58.377 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:01.499 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:58.379 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:01.502 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:58.399 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:01.522 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:58.423 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:01.520 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:58.425 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:01.522 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:58.438 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:01.561 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:58.439 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:01.562 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:58.440 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:01.537 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:07:58.457 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:01.579 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:13.421 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:16.544 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:13.429 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:16.552 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:13.446 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:16.568 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:13.471 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:16.568 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:13.474 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:16.571 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:13.493 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:16.590 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:13.498 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:16.621 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:13.500 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:16.623 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:13.521 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:16.643 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:18.538 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:21.634 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:18.538 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:21.635 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:18.549 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:21.646 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:18.552 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:21.674 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:18.555 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:21.678 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:18.568 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:21.691 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:18.603 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:21.726 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:18.608 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:21.731 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:18.619 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:21.742 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:29.943 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:08:29.940+0000 10a5e640 -1 osd.7 488 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:3817 201.16 201.f18a3536 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e485) 2023-10-06T14:08:30.948 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:08:30.945+0000 10a5e640 -1 osd.7 488 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.20203.0:3817 201.16 201.f18a3536 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e485) 2023-10-06T14:08:33.502 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:36.625 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:33.505 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:36.628 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:33.527 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:36.650 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:33.554 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:36.651 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:33.558 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:36.655 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:33.577 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:36.673 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:33.577 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:36.700 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:33.580 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:36.703 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:33.606 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:36.728 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:37.860 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2023-10-06T14:08:37.865 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: hmm, no HitSet yet 2023-10-06T14:08:37.866 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: ok, hit_set contains 200:602f83fe:::foo:head 2023-10-06T14:08:37.866 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (10280 ms) 2023-10-06T14:08:37.867 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2023-10-06T14:08:37.867 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg_num = 32 2023-10-06T14:08:37.867 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 0 ls 1696601314,0 2023-10-06T14:08:37.867 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 1 ls 1696601313,0 2023-10-06T14:08:37.867 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 2 ls 1696601314,0 2023-10-06T14:08:37.868 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 3 ls 1696601314,0 2023-10-06T14:08:37.868 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 4 ls 1696601313,0 2023-10-06T14:08:37.868 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 5 ls 1696601314,0 2023-10-06T14:08:37.868 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 6 ls 1696601314,0 2023-10-06T14:08:37.868 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 7 ls 1696601314,0 2023-10-06T14:08:38.276 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 8 ls 1696601314,0 2023-10-06T14:08:38.276 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 9 ls 1696601314,0 2023-10-06T14:08:38.276 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 10 ls 1696601314,0 2023-10-06T14:08:38.276 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 11 ls 1696601313,0 2023-10-06T14:08:38.276 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 12 ls 1696601314,0 2023-10-06T14:08:38.276 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 13 ls 1696601314,0 2023-10-06T14:08:38.276 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 14 ls 1696601314,0 2023-10-06T14:08:38.277 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 15 ls 1696601314,0 2023-10-06T14:08:38.277 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 16 ls 1696601314,0 2023-10-06T14:08:38.277 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 17 ls 1696601314,0 2023-10-06T14:08:38.277 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 18 ls 1696601314,0 2023-10-06T14:08:38.277 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 19 ls 1696601314,0 2023-10-06T14:08:38.277 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 20 ls 1696601314,0 2023-10-06T14:08:38.277 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 21 ls 1696601314,0 2023-10-06T14:08:38.277 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 22 ls 1696601314,0 2023-10-06T14:08:38.277 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 23 ls 1696601314,0 2023-10-06T14:08:38.277 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 24 ls 1696601314,0 2023-10-06T14:08:38.277 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 25 ls 1696601314,0 2023-10-06T14:08:38.277 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 26 ls 1696601314,0 2023-10-06T14:08:38.278 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 27 ls 1696601314,0 2023-10-06T14:08:38.278 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 28 ls 1696601314,0 2023-10-06T14:08:38.278 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 29 ls 1696601314,0 2023-10-06T14:08:38.278 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 30 ls 1696601314,0 2023-10-06T14:08:38.278 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg 31 ls 1696601314,0 2023-10-06T14:08:38.278 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: pg_num = 32 2023-10-06T14:08:38.278 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:6cac518f:::0:head 2023-10-06T14:08:38.278 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:02547ec2:::1:head 2023-10-06T14:08:38.278 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:f905c69b:::2:head 2023-10-06T14:08:38.278 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:cfc208b3:::3:head 2023-10-06T14:08:38.278 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:d83876eb:::4:head 2023-10-06T14:08:38.278 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:b29083e3:::5:head 2023-10-06T14:08:38.279 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:c4fdafeb:::6:head 2023-10-06T14:08:38.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:5c6b0b28:::7:head 2023-10-06T14:08:38.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:bd63b0f1:::8:head 2023-10-06T14:08:38.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:e960b815:::9:head 2023-10-06T14:08:38.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:52ea6a34:::10:head 2023-10-06T14:08:38.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:89d3ae78:::11:head 2023-10-06T14:08:38.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:de5d7c5f:::12:head 2023-10-06T14:08:38.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:566253c9:::13:head 2023-10-06T14:08:38.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:62a1935d:::14:head 2023-10-06T14:08:38.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:863748b0:::15:head 2023-10-06T14:08:38.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:3958e169:::16:head 2023-10-06T14:08:38.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:4d4dabf9:::17:head 2023-10-06T14:08:38.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:8391935d:::18:head 2023-10-06T14:08:38.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:28883081:::19:head 2023-10-06T14:08:38.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:69259c59:::20:head 2023-10-06T14:08:38.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:4bdb80b7:::21:head 2023-10-06T14:08:38.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:a11c5d71:::22:head 2023-10-06T14:08:38.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:271af37b:::23:head 2023-10-06T14:08:38.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:95b121be:::24:head 2023-10-06T14:08:38.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:58d1031b:::25:head 2023-10-06T14:08:38.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:0a050783:::26:head 2023-10-06T14:08:38.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:c709704c:::27:head 2023-10-06T14:08:38.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:cbe56eaf:::28:head 2023-10-06T14:08:38.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:86b4b162:::29:head 2023-10-06T14:08:38.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:70d89383:::30:head 2023-10-06T14:08:38.430 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:dd450c7c:::31:head 2023-10-06T14:08:38.430 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:6d5729b1:::32:head 2023-10-06T14:08:38.430 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:c388f3fb:::33:head 2023-10-06T14:08:38.430 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:56cfea31:::34:head 2023-10-06T14:08:38.430 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:9dbc1bf7:::35:head 2023-10-06T14:08:38.430 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:40b74ccd:::36:head 2023-10-06T14:08:38.430 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:4d5aaf42:::37:head 2023-10-06T14:08:38.430 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:920f362c:::38:head 2023-10-06T14:08:38.430 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:6cc53222:::39:head 2023-10-06T14:08:38.431 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:9cad833f:::40:head 2023-10-06T14:08:38.431 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:1ea84d41:::41:head 2023-10-06T14:08:38.431 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:c4480ef6:::42:head 2023-10-06T14:08:38.431 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:a694361e:::43:head 2023-10-06T14:08:38.431 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:d1bd33e9:::44:head 2023-10-06T14:08:38.431 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:ddc2cd5d:::45:head 2023-10-06T14:08:38.431 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:2b782207:::46:head 2023-10-06T14:08:38.431 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:7b187fca:::47:head 2023-10-06T14:08:38.431 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:90ecdf6f:::48:head 2023-10-06T14:08:38.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:a5ed95fe:::49:head 2023-10-06T14:08:38.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:ea0eaa55:::50:head 2023-10-06T14:08:38.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:f33ef17b:::51:head 2023-10-06T14:08:38.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:a0d1b2f6:::52:head 2023-10-06T14:08:38.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:60c5229e:::53:head 2023-10-06T14:08:38.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:edcbc575:::54:head 2023-10-06T14:08:38.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:102cf253:::55:head 2023-10-06T14:08:38.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:efb7fb0b:::56:head 2023-10-06T14:08:38.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:50d0a326:::57:head 2023-10-06T14:08:38.432 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:d4dc5daf:::58:head 2023-10-06T14:08:38.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:3a130462:::59:head 2023-10-06T14:08:38.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:ec87ed71:::60:head 2023-10-06T14:08:38.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:d5bc9454:::61:head 2023-10-06T14:08:38.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:3ddfe313:::62:head 2023-10-06T14:08:38.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:7c2816b9:::63:head 2023-10-06T14:08:38.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:47e00e4d:::64:head 2023-10-06T14:08:38.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:c6410c18:::65:head 2023-10-06T14:08:38.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:b48ed237:::66:head 2023-10-06T14:08:38.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:cd63ad31:::67:head 2023-10-06T14:08:38.433 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:b179e92b:::68:head 2023-10-06T14:08:38.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:0d9f741a:::69:head 2023-10-06T14:08:38.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:6d3352ae:::70:head 2023-10-06T14:08:38.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:c6d5c19e:::71:head 2023-10-06T14:08:38.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:bc4729c3:::72:head 2023-10-06T14:08:38.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:77e930b9:::73:head 2023-10-06T14:08:38.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:0abeecfd:::74:head 2023-10-06T14:08:38.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:b7c37e15:::75:head 2023-10-06T14:08:38.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:b6378398:::76:head 2023-10-06T14:08:38.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:02bd68de:::77:head 2023-10-06T14:08:38.434 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:cc795d2d:::78:head 2023-10-06T14:08:38.435 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:630d4fea:::79:head 2023-10-06T14:08:38.435 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:e0d29ef5:::80:head 2023-10-06T14:08:38.435 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:fd6f13d2:::81:head 2023-10-06T14:08:38.435 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:606461d5:::82:head 2023-10-06T14:08:38.435 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:eadbdc43:::83:head 2023-10-06T14:08:38.435 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:8761d0bb:::84:head 2023-10-06T14:08:38.435 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:9ef0186f:::85:head 2023-10-06T14:08:38.435 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:e0d41294:::86:head 2023-10-06T14:08:38.435 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:961de695:::87:head 2023-10-06T14:08:38.435 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:1423148f:::88:head 2023-10-06T14:08:38.435 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:633a8fa2:::89:head 2023-10-06T14:08:38.436 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:a8653809:::90:head 2023-10-06T14:08:38.436 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:3dac8b33:::91:head 2023-10-06T14:08:38.436 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:35aad435:::92:head 2023-10-06T14:08:38.436 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:f6dcc343:::93:head 2023-10-06T14:08:38.436 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:dbbdad87:::94:head 2023-10-06T14:08:38.436 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:1cb48ce0:::95:head 2023-10-06T14:08:38.436 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:03cd461c:::96:head 2023-10-06T14:08:38.436 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:17a4ea99:::97:head 2023-10-06T14:08:38.436 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:9993c9a7:::98:head 2023-10-06T14:08:38.436 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:6394211c:::99:head 2023-10-06T14:08:38.436 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:94c7ae57:::100:head 2023-10-06T14:08:38.436 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:6fdee5bb:::101:head 2023-10-06T14:08:38.437 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:9a477fd1:::102:head 2023-10-06T14:08:38.437 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:eb850916:::103:head 2023-10-06T14:08:38.437 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:affc56b9:::104:head 2023-10-06T14:08:38.437 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:b42dc814:::105:head 2023-10-06T14:08:38.437 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:f319f8f0:::106:head 2023-10-06T14:08:38.437 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:9a40b9de:::107:head 2023-10-06T14:08:38.437 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:8b524f28:::108:head 2023-10-06T14:08:38.437 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:e3de589f:::109:head 2023-10-06T14:08:38.437 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:90f90a5b:::110:head 2023-10-06T14:08:38.437 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:a7b4f1d7:::111:head 2023-10-06T14:08:38.437 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:af51766e:::112:head 2023-10-06T14:08:38.437 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:b6f90bd1:::113:head 2023-10-06T14:08:38.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:e0261208:::114:head 2023-10-06T14:08:38.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:c9569ef7:::115:head 2023-10-06T14:08:38.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:61bebe50:::116:head 2023-10-06T14:08:38.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:fe93412b:::117:head 2023-10-06T14:08:38.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:d3d38bee:::118:head 2023-10-06T14:08:38.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:3100ba0c:::119:head 2023-10-06T14:08:38.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:d0560ada:::120:head 2023-10-06T14:08:38.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:f0ea8b35:::121:head 2023-10-06T14:08:38.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:766f231a:::122:head 2023-10-06T14:08:38.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:a07a2582:::123:head 2023-10-06T14:08:38.438 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:bd7c6b3a:::124:head 2023-10-06T14:08:38.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:fb2ddaff:::125:head 2023-10-06T14:08:38.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:4408e1fe:::126:head 2023-10-06T14:08:38.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:ee1df7a7:::127:head 2023-10-06T14:08:38.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:c3002909:::128:head 2023-10-06T14:08:38.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:4f48ffa9:::129:head 2023-10-06T14:08:38.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:edf38733:::130:head 2023-10-06T14:08:38.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:c08425c0:::131:head 2023-10-06T14:08:38.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:5f902d98:::132:head 2023-10-06T14:08:38.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:41ea2c93:::133:head 2023-10-06T14:08:38.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:813cee13:::134:head 2023-10-06T14:08:38.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:0131818d:::135:head 2023-10-06T14:08:38.439 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:26ba5a85:::136:head 2023-10-06T14:08:38.440 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:381b8a5a:::137:head 2023-10-06T14:08:38.440 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:28797e47:::138:head 2023-10-06T14:08:38.440 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:bfca7f22:::139:head 2023-10-06T14:08:38.440 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:36807075:::140:head 2023-10-06T14:08:38.440 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:80b03975:::141:head 2023-10-06T14:08:38.440 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:5c15709b:::142:head 2023-10-06T14:08:38.440 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:f39ea15e:::143:head 2023-10-06T14:08:38.440 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:ea992956:::144:head 2023-10-06T14:08:38.440 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:48887b1c:::145:head 2023-10-06T14:08:38.440 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:9f24a9dd:::146:head 2023-10-06T14:08:38.440 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:987f100b:::147:head 2023-10-06T14:08:38.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:d2dd3581:::148:head 2023-10-06T14:08:38.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:7fed1808:::149:head 2023-10-06T14:08:38.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:c80b70e9:::150:head 2023-10-06T14:08:38.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:85ed90f9:::151:head 2023-10-06T14:08:38.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:36428b24:::152:head 2023-10-06T14:08:38.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:d044c34a:::153:head 2023-10-06T14:08:38.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:7c18bf58:::154:head 2023-10-06T14:08:38.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:d1c21232:::155:head 2023-10-06T14:08:38.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:a7a3c575:::156:head 2023-10-06T14:08:38.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:87da0633:::157:head 2023-10-06T14:08:38.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:d5ac3822:::158:head 2023-10-06T14:08:38.441 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:3f20522d:::159:head 2023-10-06T14:08:38.442 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:6ca26563:::160:head 2023-10-06T14:08:38.442 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:532ce135:::161:head 2023-10-06T14:08:38.442 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:c78863e6:::162:head 2023-10-06T14:08:38.517 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:41.640 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:38.588 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:41.643 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:38.589 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:41.656 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:38.589 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:41.662 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:38.589 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:41.664 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:38.589 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:41.674 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:38.589 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:41.693 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:38.589 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:41.696 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:38.590 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:41.711 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:44.670 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:47.793 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:44.671 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:47.794 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:44.697 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:47.820 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:44.723 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:47.819 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:44.724 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:47.821 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:44.735 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:47.858 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:44.900 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:47.855 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:44.900 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:47.862 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:44.900 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:47.880 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:53.639 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:56.762 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:53.641 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:56.764 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:53.659 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:56.782 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:53.679 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:56.775 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:53.682 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:56.779 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:53.683 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:56.805 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:53.686 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:56.809 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:53.701 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:56.798 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:53.707 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:56.830 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:54.433 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:57.530 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:54.434 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:57.531 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:54.443 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:16:57.540 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:54.505 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:57.627 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:54.506 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:57.629 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:54.519 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:16:57.642 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:54.522 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:57.645 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:54.523 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:57.646 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:08:54.538 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:16:57.658 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:13.629 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:16.752 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:13.687 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:16.810 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:13.687 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:16.810 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:13.691 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:16.814 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:13.717 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:16.840 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:13.722 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:16.845 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:13.735 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:16.832 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:13.736 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:16.833 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:13.755 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:16.852 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:14.398 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:17.492 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:14.398 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:17.493 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:14.404 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:17.527 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:14.405 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:17.502 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:14.405 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:17.528 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:14.408 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:17.531 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:14.410 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:17.533 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:14.416 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:17.539 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:14.419 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:17.542 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:28.733 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:31.856 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:28.735 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:31.858 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:28.758 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:31.881 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:28.779 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:31.876 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:28.781 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:31.878 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:28.782 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:31.905 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:28.785 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:31.908 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:28.804 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:31.900 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:28.810 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:31.932 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:33.886 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:37.008 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:33.895 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:36.992 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:33.896 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:37.018 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:33.898 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:36.995 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:33.941 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:37.064 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:33.956 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:37.079 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:33.958 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:37.081 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:33.976 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:37.072 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:33.982 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:37.104 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:43.803 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:46.926 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:43.805 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:46.928 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:43.826 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:46.949 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:43.850 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:46.973 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:43.852 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:46.949 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:43.853 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:46.976 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:43.854 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:46.951 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:43.876 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:46.973 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:43.885 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:47.008 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:53.832 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:56.955 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:53.835 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:56.958 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:53.852 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:17:56.975 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:53.884 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:56.980 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:53.885 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:57.007 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:53.885 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:56.982 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:53.886 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:57.009 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:53.898 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:17:56.995 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:09:53.900 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:17:57.023 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:00.424 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:7462ddf6:::163:head 2023-10-06T14:10:00.424 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:5d165639:::164:head 2023-10-06T14:10:00.424 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:f43765fc:::165:head 2023-10-06T14:10:00.424 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:b4c720e9:::166:head 2023-10-06T14:10:00.424 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:e694b040:::167:head 2023-10-06T14:10:00.425 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:afa38db2:::168:head 2023-10-06T14:10:00.425 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:77ba9f53:::169:head 2023-10-06T14:10:00.425 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:87495034:::170:head 2023-10-06T14:10:00.425 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:7c96bf0e:::171:head 2023-10-06T14:10:00.425 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:dbe346cc:::172:head 2023-10-06T14:10:00.425 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:e943ec24:::173:head 2023-10-06T14:10:00.425 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:f97a9c0c:::174:head 2023-10-06T14:10:00.425 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:6f26e74d:::175:head 2023-10-06T14:10:00.425 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:4f95e106:::176:head 2023-10-06T14:10:00.425 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:0e6f2f8f:::177:head 2023-10-06T14:10:00.425 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:05db05f1:::178:head 2023-10-06T14:10:00.425 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:38a78d66:::179:head 2023-10-06T14:10:00.426 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:d095610b:::180:head 2023-10-06T14:10:00.426 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:a1a9d709:::181:head 2023-10-06T14:10:00.426 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:1e5d39db:::182:head 2023-10-06T14:10:00.426 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:f7df4fb9:::183:head 2023-10-06T14:10:00.426 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:03a7f161:::184:head 2023-10-06T14:10:00.426 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:ba70721e:::185:head 2023-10-06T14:10:00.426 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:28e5662d:::186:head 2023-10-06T14:10:00.426 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:973d52de:::187:head 2023-10-06T14:10:00.426 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:4303eb1c:::188:head 2023-10-06T14:10:00.426 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:b990b48e:::189:head 2023-10-06T14:10:00.426 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:29b8165b:::190:head 2023-10-06T14:10:00.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:3547f197:::191:head 2023-10-06T14:10:00.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:7e260936:::192:head 2023-10-06T14:10:00.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:1abec7b1:::193:head 2023-10-06T14:10:00.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:10fdda93:::194:head 2023-10-06T14:10:00.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:15817eea:::195:head 2023-10-06T14:10:00.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:770bab57:::196:head 2023-10-06T14:10:00.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:ed9e13e7:::197:head 2023-10-06T14:10:00.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:71471a8f:::198:head 2023-10-06T14:10:00.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: checking for 201:10fb1d02:::199:head 2023-10-06T14:10:00.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (55657 ms) 2023-10-06T14:10:00.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2023-10-06T14:10:00.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696601336,0 2023-10-06T14:10:00.427 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: first is 1696601336 2023-10-06T14:10:00.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696601336,0 2023-10-06T14:10:00.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696601336,0 2023-10-06T14:10:00.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696601336,1696601338,1696601339,0 2023-10-06T14:10:00.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696601336,1696601338,1696601339,0 2023-10-06T14:10:00.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696601336,1696601338,1696601339,0 2023-10-06T14:10:00.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696601336,1696601338,1696601339,1696601341,1696601342,0 2023-10-06T14:10:00.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696601336,1696601338,1696601339,1696601341,1696601342,0 2023-10-06T14:10:00.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696601336,1696601338,1696601339,1696601341,1696601342,0 2023-10-06T14:10:00.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696601336,1696601338,1696601339,1696601341,1696601342,1696601344,1696601345,0 2023-10-06T14:10:00.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696601336,1696601338,1696601339,1696601341,1696601342,1696601344,1696601345,0 2023-10-06T14:10:00.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696601336,1696601338,1696601339,1696601341,1696601342,1696601344,1696601345,0 2023-10-06T14:10:00.428 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696601339,1696601341,1696601342,1696601344,1696601345,1696601347,1696601348,0 2023-10-06T14:10:00.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: first now 1696601339, trimmed 2023-10-06T14:10:00.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (35619 ms) 2023-10-06T14:10:00.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2023-10-06T14:10:00.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: foo0 2023-10-06T14:10:00.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: verifying foo0 is eventually promoted 2023-10-06T14:10:00.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (16309 ms) 2023-10-06T14:10:00.429 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2023-10-06T14:10:08.888 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:12.011 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:08.890 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:12.013 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:08.915 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:12.037 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:08.933 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:12.030 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:08.935 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:12.032 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:08.936 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:12.059 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:08.939 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:12.062 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:08.958 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:12.055 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:08.967 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:12.090 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:13.897 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:17.020 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:13.899 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:17.022 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:13.912 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:17.035 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:13.945 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:17.041 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:13.946 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:17.043 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:13.950 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:17.073 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:13.952 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:17.074 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:13.956 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:17.053 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:13.963 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:17.085 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:23.938 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:27.061 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:23.940 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:27.063 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:23.959 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:27.082 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:23.989 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:27.086 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:23.990 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:27.087 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:23.992 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:27.115 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:23.994 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:27.117 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:24.009 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:27.106 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:24.012 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:27.135 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:34.093 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:37.215 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:34.158 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:37.281 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:34.182 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:37.305 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:34.225 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:37.348 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:34.229 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:37.326 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:34.229 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:37.352 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:34.235 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:37.332 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:34.256 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:37.379 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:34.257 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:37.354 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:38.949 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:42.071 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:38.999 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:42.122 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:39.017 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:42.140 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:39.041 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:42.138 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:39.043 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:42.140 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:39.056 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:42.153 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:39.072 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:42.194 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:39.074 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:42.197 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:39.094 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:42.217 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:54.042 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:57.165 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:54.047 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:57.170 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:54.092 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:18:57.214 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:54.095 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:57.192 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:54.099 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:57.197 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:54.109 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:57.232 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:54.113 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:57.236 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:54.129 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:18:57.226 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:54.143 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:18:57.266 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:57.963 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:01.086 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:57.965 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:01.087 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:57.980 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:01.102 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:58.010 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:01.106 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:58.013 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:01.110 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:58.016 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:01.139 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:58.018 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:01.140 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:58.033 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:01.155 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:58.035 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:01.132 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:59.196 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:02.319 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:59.198 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:02.321 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:59.208 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:02.331 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:59.241 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:02.338 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:59.246 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:02.369 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:59.248 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:02.345 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:59.249 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:02.372 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:59.258 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:02.381 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:10:59.261 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:02.358 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:14.116 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:17.238 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:14.117 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:17.240 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:14.135 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:17.258 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:14.157 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:17.254 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:14.163 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:17.260 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:14.168 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:17.291 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:14.172 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:17.295 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:14.184 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:17.281 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:14.188 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:17.311 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:19.133 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:22.255 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:19.135 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:22.258 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:19.147 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:22.270 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:19.180 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:22.277 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:19.183 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:22.280 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:19.184 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:22.307 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:19.186 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:22.309 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:19.196 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:22.293 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:19.198 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:22.321 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:34.173 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:37.296 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:34.175 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:37.298 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:34.190 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:37.312 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:34.221 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:37.315 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:34.224 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:37.321 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:34.224 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:37.347 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:34.228 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:37.349 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:34.406 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:37.335 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:34.407 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:37.364 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:39.187 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:42.310 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:39.189 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:42.312 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:39.201 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:42.324 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:39.231 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:42.328 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:39.235 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:42.332 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:39.239 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:42.362 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:39.241 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:42.364 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:39.246 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:42.343 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:39.251 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:42.374 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:54.231 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:57.353 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:54.232 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:57.355 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:54.248 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:19:57.371 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:54.274 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:57.371 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:54.276 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:57.373 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:54.284 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:57.407 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:54.285 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:57.408 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:54.293 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:19:57.390 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:54.308 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:19:57.431 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:59.246 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:02.366 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:59.246 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:02.369 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:59.260 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:02.383 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:59.290 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:02.387 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:59.294 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:02.391 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:59.298 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:02.420 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:59.301 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:02.424 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:59.305 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:02.402 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:11:59.313 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:02.436 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:04.467 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:07.590 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:04.471 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:07.594 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:04.497 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:07.619 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:04.527 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:07.623 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:04.529 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:07.626 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:04.532 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:07.655 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:04.534 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:07.657 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:04.548 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:07.645 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:04.553 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:07.676 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:09.456 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:12.579 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:09.461 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:12.583 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:09.484 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:12.607 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:09.509 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:12.606 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:09.510 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:12.633 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:09.512 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:12.609 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:09.517 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:12.636 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:09.526 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:12.623 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:09.528 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:12.651 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:19.667 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:22.790 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:19.669 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:22.792 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:19.686 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:22.809 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:19.715 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:22.812 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:19.716 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:22.813 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:19.722 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:22.845 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:19.724 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:22.847 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:19.733 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:22.829 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:19.742 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:22.865 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:24.533 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:27.656 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:24.539 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:27.662 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:24.551 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:27.673 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:24.576 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:27.672 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:24.579 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:27.676 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:24.580 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:27.703 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:24.584 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:27.707 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:24.590 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:27.687 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:24.595 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:27.718 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:39.684 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:42.806 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:39.685 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:42.808 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:39.716 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:42.838 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:39.737 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:42.834 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:39.739 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:42.836 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:39.758 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:42.855 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:39.759 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:42.881 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:39.763 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:42.886 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:39.783 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:42.906 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:40.192 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:43.289 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:40.238 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:43.335 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:40.251 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:43.374 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:40.252 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:43.375 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:40.253 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:43.350 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:40.262 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:43.384 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:40.293 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:43.415 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:40.294 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:43.417 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:40.303 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:43.425 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:54.795 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:57.917 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:54.800 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:57.923 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:54.801 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:57.898 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:54.805 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:57.902 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:54.828 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:20:57.925 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:54.833 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:20:57.956 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:54.876 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:57.999 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:54.878 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:58.001 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:54.907 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:20:58.030 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:59.863 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:02.986 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:59.867 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:02.990 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:59.878 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:03.001 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:59.912 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:03.009 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:59.913 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:03.010 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:59.915 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:03.037 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:59.916 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:03.039 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:59.923 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:03.020 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:12:59.927 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:03.050 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:09.843 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:12.966 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:09.850 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:12.973 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:09.887 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:13.010 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:09.912 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:13.009 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:09.918 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:13.015 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:09.922 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:13.045 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:09.930 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:13.052 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:09.949 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:13.046 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:09.962 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:13.085 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:19.837 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:22.960 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:19.840 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:22.963 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:19.859 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:22.982 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:19.885 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:22.982 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:19.888 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:22.985 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:19.888 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:23.011 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:19.889 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:23.012 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:19.908 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:23.005 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:19.909 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:23.032 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:24.848 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:27.971 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:24.852 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:27.975 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:24.865 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:27.988 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:24.896 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:27.993 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:24.899 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:27.996 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:24.902 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:28.025 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:24.906 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:28.028 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:24.910 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:28.006 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:24.917 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:28.040 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:36.572 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:39.695 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:36.584 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:39.698 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:36.601 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:39.724 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:36.632 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:39.729 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:36.633 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:39.731 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:36.637 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:39.760 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:36.639 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:39.762 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:36.650 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:39.747 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:36.657 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:39.780 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:39.967 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:43.090 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:39.973 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:43.096 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:39.991 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:43.114 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:40.031 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:43.128 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:40.042 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:43.139 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:40.051 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:43.174 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:40.057 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:43.179 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:40.059 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:43.156 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:40.071 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:43.194 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:45.091 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:48.213 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:45.093 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:48.216 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:45.106 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:21:48.229 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:45.135 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:48.232 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:45.137 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:48.234 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:45.149 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:21:48.246 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:45.153 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:48.276 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:45.154 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:48.277 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:13:45.167 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:21:48.290 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:00.142 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:03.265 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:00.145 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:03.268 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:00.166 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:03.289 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:00.192 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:03.289 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:00.194 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:03.291 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:00.201 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:03.323 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:00.204 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:03.327 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:00.215 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:03.311 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:00.226 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:03.349 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:00.700 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:14:00.678+0000 24e90640 -1 snap_mapper.add_oid found existing snaps mapped on 77:5ddb27bd:test-rados-api-smithi019-91898-76::foo:28, removing 2023-10-06T14:14:00.707 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:14:00.699+0000 27a96640 -1 snap_mapper.add_oid found existing snaps mapped on 77:5ddb27bd:test-rados-api-smithi019-91898-76::foo:28, removing 2023-10-06T14:14:05.169 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:08.292 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:05.173 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:08.296 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:05.189 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:08.312 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:05.217 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:08.313 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:05.218 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:08.315 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:05.221 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:08.344 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:05.222 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:08.345 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:05.233 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:08.330 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:05.238 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:08.361 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:05.756 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:14:05.734+0000 2428f640 -1 snap_mapper.add_oid found existing snaps mapped on 77:b6082442:test-rados-api-smithi019-91898-78::foo:29, removing 2023-10-06T14:14:05.763 INFO:tasks.ceph.osd.3.smithi019.stderr:2023-10-06T14:14:05.754+0000 2d6a1640 -1 snap_mapper.add_oid found existing snaps mapped on 77:b6082442:test-rados-api-smithi019-91898-78::foo:29, removing 2023-10-06T14:14:15.077 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:18.200 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:15.081 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:18.204 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:15.100 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:18.222 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:15.124 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:18.221 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:15.127 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:18.224 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:15.145 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:18.242 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:15.150 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:18.273 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:15.152 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:18.275 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:15.173 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:18.296 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:30.141 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:33.264 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:30.144 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:33.267 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:30.158 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:33.281 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:30.192 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:33.288 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:30.193 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:33.290 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:30.196 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:33.319 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:30.197 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:33.320 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:30.208 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:33.305 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:30.211 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:33.334 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:32.955 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:14:32.942+0000 24e90640 -1 snap_mapper.add_oid found existing snaps mapped on 219:981f26ba:test-rados-api-smithi019-91898-80::foo:2, removing 2023-10-06T14:14:32.957 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:14:32.951+0000 2b29d640 -1 snap_mapper.add_oid found existing snaps mapped on 219:981f26ba:test-rados-api-smithi019-91898-80::foo:2, removing 2023-10-06T14:14:35.168 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:38.290 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:35.170 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:38.293 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:35.182 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:38.304 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:35.215 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:38.338 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:35.216 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:38.339 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:35.223 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:38.320 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:35.226 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:38.323 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:35.234 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:38.357 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:35.239 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:38.336 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:50.176 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:53.299 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:50.228 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:53.350 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:50.259 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:53.382 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:50.271 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:53.368 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:50.274 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:53.371 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:50.278 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:53.400 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:50.282 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:53.403 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:50.303 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:53.400 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:50.311 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:53.434 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:50.956 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:54.053 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:50.957 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:54.054 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:50.966 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:22:54.063 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:51.148 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:54.271 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:51.149 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:54.272 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:51.159 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:22:54.282 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:51.217 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:54.340 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:51.218 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:54.341 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:14:51.230 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:22:54.353 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:05.288 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:08.411 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:05.290 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:08.413 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:05.315 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:08.437 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:05.337 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:08.434 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:05.339 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:08.436 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:05.339 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:08.462 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:05.343 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:08.465 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:05.363 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:08.460 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:05.367 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:08.490 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:10.294 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:13.416 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:10.297 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:13.420 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:10.308 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:13.431 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:10.338 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:13.435 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:10.342 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:13.439 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:10.344 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:13.467 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:10.347 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:13.470 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:10.352 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:13.449 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:10.361 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:13.484 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:30.345 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:33.466 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:30.348 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:33.471 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:30.367 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:33.490 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:30.396 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:33.493 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:30.397 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:33.494 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:30.399 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:33.522 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:30.402 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:33.525 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:30.414 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:33.511 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:30.425 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:33.548 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:31.037 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:34.134 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:31.038 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:34.135 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:31.047 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:34.144 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:31.072 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:34.194 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:31.073 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:34.196 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:31.082 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:34.204 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:31.133 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:34.256 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:31.134 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:34.257 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:31.143 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:34.265 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:50.407 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:53.530 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:50.410 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:53.533 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:50.437 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:53.560 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:50.458 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:53.555 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:50.459 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:53.556 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:50.469 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:53.592 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:50.471 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:53.594 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:50.476 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:53.573 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:50.490 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:53.612 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:51.063 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:54.160 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:51.064 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:54.161 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:51.073 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:23:54.168 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:51.077 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:54.200 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:51.078 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:54.201 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:51.086 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:23:54.209 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:51.136 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:54.259 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:51.137 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:54.260 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:15:51.148 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:23:54.271 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:10.464 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:13.587 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:10.465 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:13.588 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:10.482 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:13.605 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:10.512 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:13.635 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:10.513 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:13.636 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:10.515 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:13.612 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:10.516 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:13.613 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:10.532 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:13.629 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:10.538 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:13.660 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.125 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:14.221 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.125 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:14.222 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.128 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:14.251 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.129 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:14.252 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.133 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:14.230 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.139 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:14.262 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.196 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:14.319 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.197 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:14.320 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.205 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:14.328 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.891 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:15.014 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.915 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:15.025 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.915 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:15.026 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.915 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:15.038 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.942 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:15.038 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.942 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:15.064 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.943 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:15.040 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.954 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:15.077 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:11.955 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:15.052 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:18.592 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:16:18.586+0000 10a5e640 -1 osd.4 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:56.633680+0000 front 2023-10-06T14:14:56.626943+0000 (oldest deadline 2023-10-06T14:16:18.310339+0000) 2023-10-06T14:16:18.834 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:16:18.831+0000 10a5e640 -1 osd.7 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:57.558906+0000 front 2023-10-06T14:14:57.561884+0000 (oldest deadline 2023-10-06T14:16:18.051369+0000) 2023-10-06T14:16:19.312 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:16:19.308+0000 10a5e640 -1 osd.2 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:55.606055+0000 front 2023-10-06T14:14:55.603103+0000 (oldest deadline 2023-10-06T14:16:19.067368+0000) 2023-10-06T14:16:19.628 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:16:19.624+0000 10a5e640 -1 osd.4 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:56.633680+0000 front 2023-10-06T14:14:56.626943+0000 (oldest deadline 2023-10-06T14:16:18.310339+0000) 2023-10-06T14:16:19.830 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:16:19.827+0000 10a5e640 -1 osd.7 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:57.558906+0000 front 2023-10-06T14:14:57.561884+0000 (oldest deadline 2023-10-06T14:16:18.051369+0000) 2023-10-06T14:16:19.944 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:16:19.940+0000 10a5e640 -1 osd.0 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:57.135962+0000 front 2023-10-06T14:14:57.133816+0000 (oldest deadline 2023-10-06T14:16:19.417609+0000) 2023-10-06T14:16:20.300 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:16:20.291+0000 10a5e640 -1 osd.2 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:55.606055+0000 front 2023-10-06T14:14:55.603103+0000 (oldest deadline 2023-10-06T14:16:19.067368+0000) 2023-10-06T14:16:20.623 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:16:20.619+0000 10a5e640 -1 osd.4 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:56.633680+0000 front 2023-10-06T14:14:56.626943+0000 (oldest deadline 2023-10-06T14:16:18.310339+0000) 2023-10-06T14:16:20.798 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:16:20.795+0000 10a5e640 -1 osd.7 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:57.558906+0000 front 2023-10-06T14:14:57.561884+0000 (oldest deadline 2023-10-06T14:16:18.051369+0000) 2023-10-06T14:16:20.963 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:16:20.959+0000 10a5e640 -1 osd.0 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:57.135962+0000 front 2023-10-06T14:14:57.133816+0000 (oldest deadline 2023-10-06T14:16:19.417609+0000) 2023-10-06T14:16:21.018 INFO:tasks.ceph.osd.3.smithi019.stderr:2023-10-06T14:16:21.015+0000 10a5e640 -1 osd.3 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:57.440207+0000 front 2023-10-06T14:14:57.442140+0000 (oldest deadline 2023-10-06T14:16:20.927468+0000) 2023-10-06T14:16:21.256 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:16:21.253+0000 10a5e640 -1 osd.2 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:55.606055+0000 front 2023-10-06T14:14:55.603103+0000 (oldest deadline 2023-10-06T14:16:19.067368+0000) 2023-10-06T14:16:21.643 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:16:21.639+0000 10a5e640 -1 osd.4 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:56.633680+0000 front 2023-10-06T14:14:56.626943+0000 (oldest deadline 2023-10-06T14:16:18.310339+0000) 2023-10-06T14:16:21.761 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:16:21.757+0000 10a5e640 -1 osd.7 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:57.558906+0000 front 2023-10-06T14:14:57.561884+0000 (oldest deadline 2023-10-06T14:16:18.051369+0000) 2023-10-06T14:16:21.937 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:16:21.930+0000 1125f640 -1 osd.1 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:55.055452+0000 front 2023-10-06T14:14:55.059372+0000 (oldest deadline 2023-10-06T14:16:20.947229+0000) 2023-10-06T14:16:22.034 INFO:tasks.ceph.osd.3.smithi019.stderr:2023-10-06T14:16:22.031+0000 10a5e640 -1 osd.3 650 heartbeat_check: no reply from 172.21.15.83:6822 osd.5 since back 2023-10-06T14:14:57.440207+0000 front 2023-10-06T14:14:57.442140+0000 (oldest deadline 2023-10-06T14:16:20.927468+0000) 2023-10-06T14:16:25.518 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:28.641 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:25.520 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:28.643 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:25.538 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:28.661 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:25.566 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:28.689 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:25.569 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:28.692 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:25.586 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:28.683 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:25.589 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:28.712 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:25.592 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:28.689 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:25.618 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:28.715 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:30.214 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ProxyRead (20605 ms) 2023-10-06T14:16:30.214 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2023-10-06T14:16:30.214 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (33309 ms) 2023-10-06T14:16:30.214 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2023-10-06T14:16:30.214 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3511 ms) 2023-10-06T14:16:30.214 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2023-10-06T14:16:30.214 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.214 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (4697 ms) 2023-10-06T14:16:30.215 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2023-10-06T14:16:30.215 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (4535 ms) 2023-10-06T14:16:30.215 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2023-10-06T14:16:30.215 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.215 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (4716 ms) 2023-10-06T14:16:30.215 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2023-10-06T14:16:30.215 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.215 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (72751 ms) 2023-10-06T14:16:30.215 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2023-10-06T14:16:30.215 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.215 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (40593 ms) 2023-10-06T14:16:30.215 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2023-10-06T14:16:30.216 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.216 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (17376 ms) 2023-10-06T14:16:30.216 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2023-10-06T14:16:30.216 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.216 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (5569 ms) 2023-10-06T14:16:30.216 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2023-10-06T14:16:30.216 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.216 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (4519 ms) 2023-10-06T14:16:30.216 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2023-10-06T14:16:30.216 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.216 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (26778 ms) 2023-10-06T14:16:30.216 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2023-10-06T14:16:30.217 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.217 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (16284 ms) 2023-10-06T14:16:30.217 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2023-10-06T14:16:30.217 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.217 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (6895 ms) 2023-10-06T14:16:30.217 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2023-10-06T14:16:30.217 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.217 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (5680 ms) 2023-10-06T14:16:30.217 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2023-10-06T14:16:30.217 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.217 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: waiting for scrubs... 2023-10-06T14:16:30.217 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: done waiting 2023-10-06T14:16:30.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (26085 ms) 2023-10-06T14:16:30.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2023-10-06T14:16:30.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (11244 ms) 2023-10-06T14:16:30.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2023-10-06T14:16:30.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (10385 ms) 2023-10-06T14:16:30.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2023-10-06T14:16:30.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:16:30.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (92433 ms) 2023-10-06T14:16:30.218 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2023-10-06T14:16:30.688 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:33.811 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:30.743 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:33.840 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:30.746 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:33.843 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:30.761 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:33.858 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:30.762 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:33.885 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:30.793 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:33.915 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:30.842 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:33.965 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:30.846 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:33.969 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:30.867 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:33.990 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:31.261 INFO:tasks.ceph.osd.5.smithi083.stderr:2023-10-06T14:16:31.258+0000 12261640 -1 osd.5 657 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T14:16:45.692 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:48.815 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:45.694 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:48.817 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:45.727 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:48.850 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:45.754 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:48.851 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:45.755 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:48.878 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:45.756 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:48.853 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:45.759 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:48.882 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:45.790 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:48.887 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:45.791 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:48.914 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:50.631 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:53.754 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:50.634 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:53.757 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:50.650 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:24:53.773 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:50.682 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:53.778 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:50.683 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:53.780 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:50.685 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:53.808 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:50.690 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:53.813 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:50.696 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:24:53.793 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:16:50.704 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:24:53.827 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:00.822 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:03.944 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:00.824 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:03.947 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:00.852 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:03.975 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:00.878 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:03.975 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:00.879 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:03.976 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:00.888 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:04.010 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:00.891 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:04.014 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:00.898 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:03.995 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:00.914 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:04.037 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:10.844 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:13.966 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:10.846 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:13.969 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:10.861 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:13.983 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:10.894 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:13.991 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:10.894 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:14.017 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:10.895 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:13.992 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:10.896 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:14.019 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:10.908 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:14.031 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:10.909 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:14.006 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:20.982 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:24.104 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:20.985 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:24.108 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:21.011 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:24.134 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:21.051 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:24.148 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:21.051 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:24.174 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:21.059 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:24.156 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:21.060 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:24.183 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:21.081 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:24.178 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:21.085 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:24.208 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:30.908 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:34.031 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:30.910 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:34.033 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:30.955 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:34.051 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:30.956 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:34.053 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:30.957 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:34.080 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:30.962 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:34.085 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:30.962 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:34.085 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:30.996 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:34.092 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:31.007 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:34.130 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:32.561 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:17:32.553+0000 2468f640 -1 snap_mapper.add_oid found existing snaps mapped on 77:fb7b79f2:test-rados-api-smithi019-91898-97::foo:31, removing 2023-10-06T14:17:32.577 INFO:tasks.ceph.osd.5.smithi083.stderr:2023-10-06T14:17:32.567+0000 2e2a3640 -1 snap_mapper.add_oid found existing snaps mapped on 77:fb7b79f2:test-rados-api-smithi019-91898-97::foo:31, removing 2023-10-06T14:17:35.893 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:39.016 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:35.894 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:39.017 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:35.910 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:39.033 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:35.938 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:39.035 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:35.955 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:39.052 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:35.967 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:39.089 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:35.968 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:39.091 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:35.970 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:39.067 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:35.986 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:39.109 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:51.143 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:54.266 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:51.145 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:54.268 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:51.177 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:25:54.300 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:51.196 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:54.293 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:51.197 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:54.294 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:51.212 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:54.334 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:51.212 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:54.336 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:51.226 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:25:54.323 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:17:51.257 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:25:54.380 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:06.046 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:09.168 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:06.047 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:09.170 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:06.075 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:09.198 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:06.092 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:09.188 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:06.098 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:09.221 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:06.100 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:09.223 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:06.105 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:09.202 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:06.133 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:09.256 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:06.133 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:09.230 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:11.204 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:14.326 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:11.206 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:14.329 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:11.226 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:14.349 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:11.259 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:14.356 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:11.265 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:14.358 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:11.265 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:14.384 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:11.266 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:14.389 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:11.280 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:14.377 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:11.285 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:14.408 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:18.824 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:21.947 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:18.826 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:21.949 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:18.835 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:21.931 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:18.837 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:21.934 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:18.843 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:21.965 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:18.859 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:21.980 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:18.926 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:22.023 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:18.934 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:22.057 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:18.943 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:22.066 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:26.078 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:29.200 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:26.128 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:29.251 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:26.153 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:29.274 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:26.177 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:29.274 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:26.179 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:29.275 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:26.184 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:29.306 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:26.187 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:29.310 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:26.197 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:29.294 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:26.215 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:29.337 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:31.096 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:34.218 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:31.142 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:34.239 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:31.143 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:34.240 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:31.147 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:34.270 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:31.157 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:34.254 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:31.169 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:34.292 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:31.200 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:34.323 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:31.203 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:34.326 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:31.226 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:34.349 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:41.184 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:44.306 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:41.186 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:44.309 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:41.213 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:44.335 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:41.235 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:44.331 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:41.235 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:44.332 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:41.237 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:44.360 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:41.241 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:44.364 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:41.262 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:44.358 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:41.266 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:44.389 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:51.433 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:54.556 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:51.439 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:54.562 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:51.475 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:54.598 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:51.501 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:54.597 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:51.502 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:54.599 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:51.506 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:54.629 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:51.509 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:54.632 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:51.531 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:54.628 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:51.541 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:54.664 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:52.184 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:55.280 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:52.185 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:55.282 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:52.195 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:55.292 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:52.297 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:55.419 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:52.298 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:55.421 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:52.307 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:55.429 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:52.369 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:55.491 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:52.370 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:55.493 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:52.383 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:55.506 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:56.694 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:59.816 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:56.753 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:59.875 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:56.762 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:59.859 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:56.766 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:59.863 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:56.780 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:26:59.902 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:56.790 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:26:59.886 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:56.819 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:59.942 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:56.829 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:59.952 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:18:56.856 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:26:59.979 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:11.701 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:14.823 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:11.702 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:14.825 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:11.732 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:14.854 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:11.755 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:14.878 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:11.756 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:14.879 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:11.759 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:14.856 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:11.761 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:14.858 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:11.789 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:14.885 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:11.800 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:14.923 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:26.677 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:29.800 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:26.681 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:29.804 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:26.704 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:29.827 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:26.722 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:29.819 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:26.725 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:29.822 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:26.732 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:29.855 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:26.733 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:29.856 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:26.748 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:29.845 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:26.757 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:29.880 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:31.672 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:34.794 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:31.677 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:34.800 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:31.687 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:34.810 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:31.722 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:34.819 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:31.724 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:34.821 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:31.725 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:34.848 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:31.727 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:34.850 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:31.736 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:34.833 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:31.739 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:34.862 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:51.782 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:54.905 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:51.785 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:54.908 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:51.822 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:54.945 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:51.829 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:54.926 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:51.831 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:54.928 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:51.847 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:54.969 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:51.849 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:54.972 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:51.864 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:54.960 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:51.885 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:55.007 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:52.437 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:55.534 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:52.441 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:55.538 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:52.500 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:27:55.597 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:52.654 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:55.777 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:52.661 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:55.784 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:52.680 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:27:55.803 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:52.694 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:55.817 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:52.696 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:55.819 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:19:52.708 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:27:55.831 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:06.974 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:10.097 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:06.980 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:10.102 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:07.021 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:10.144 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:07.026 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:10.123 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:07.029 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:10.125 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:07.033 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:10.156 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:07.037 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:10.160 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:07.069 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:10.192 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:07.073 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:10.170 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:11.858 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:14.981 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:11.861 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:14.984 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:11.874 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:14.997 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:11.912 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:15.034 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:11.913 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:15.009 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:11.914 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:15.011 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:11.914 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:15.037 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:11.924 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:15.020 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:11.929 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:15.051 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:24.552 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:27.674 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:24.556 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:27.679 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:24.584 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:27.706 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:24.609 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:27.706 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:24.610 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:27.707 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:24.628 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:27.724 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:24.628 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:27.751 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:24.630 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:27.753 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:24.649 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:27.772 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:26.911 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:30.034 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:26.913 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:30.036 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:26.924 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:30.047 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:26.959 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:30.056 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:26.960 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:30.057 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:26.965 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:30.088 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:26.969 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:30.066 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:26.981 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:30.104 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:26.992 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:30.115 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:31.996 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:35.118 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:32.000 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:35.123 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:32.017 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:35.140 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:32.045 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:35.142 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:32.048 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:35.145 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:32.059 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:35.156 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:32.069 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:35.192 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:32.072 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:35.195 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:32.088 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:35.211 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:36.426 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:36.423+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:37.442 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:37.438+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:38.493 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:38.489+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:39.467 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:39.464+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:40.463 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:40.459+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:41.499 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:41.495+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:41.989 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:45.112 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:41.991 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:45.114 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:42.003 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:28:45.126 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:42.035 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:45.132 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:42.040 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:45.163 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:42.040 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:45.137 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:42.042 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:45.165 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:42.052 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:28:45.149 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:42.055 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:28:45.178 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:42.545 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:42.542+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:43.501 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:43.497+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:44.469 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:44.466+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:45.513 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:45.510+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:46.497 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:46.494+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:47.459 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:47.455+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:48.480 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:48.477+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:49.509 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:49.507+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:50.544 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:50.542+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:51.518 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:51.516+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:52.490 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:52.488+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:53.473 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:53.470+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:54.436 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:54.434+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:55.396 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:55.391+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:56.404 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:56.401+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:57.091 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:00.214 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.093 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:00.215 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.109 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:00.231 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.136 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:00.233 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.137 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:00.234 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.143 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:00.266 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.154 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:00.250 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.160 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:00.283 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.183 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:00.305 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.405 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:57.403+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:57.721 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:00.818 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.722 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:00.819 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.729 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:00.826 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.792 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:00.915 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.793 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:00.916 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.801 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:00.924 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.818 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:00.940 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.819 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:00.941 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:57.827 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:00.950 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:20:58.417 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:58.414+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:20:59.448 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:20:59.446+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:21:00.491 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:21:00.489+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:21:01.510 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:21:01.508+0000 10a5e640 -1 osd.4 818 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:653 243.8 243.8 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e817) 2023-10-06T14:21:12.247 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:15.369 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:12.249 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:15.372 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:12.289 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:15.411 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:12.293 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:15.390 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:12.294 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:15.391 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:12.314 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:15.437 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:12.316 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:15.439 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:12.319 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:15.416 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:12.343 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:15.466 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:17.298 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:20.421 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:17.300 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:20.423 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:17.316 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:20.439 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:17.347 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:20.470 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:17.349 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:20.446 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:17.350 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:20.447 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:17.353 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:20.476 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:17.363 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:20.460 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:17.369 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:20.492 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:32.200 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:35.322 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:32.205 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:35.328 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:32.221 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:35.344 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:32.253 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:35.376 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:32.254 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:35.350 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:32.255 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:35.378 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:32.257 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:35.354 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:32.274 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:35.371 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:32.281 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:35.404 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:37.216 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:40.339 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:37.219 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:40.342 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:37.229 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:40.352 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:37.262 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:40.384 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:37.265 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:40.362 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:37.265 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:40.388 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:37.266 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:40.363 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:37.279 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:40.375 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:37.280 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:40.403 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:44.054 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:44.050+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:45.091 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:45.088+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:46.139 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:46.136+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:47.159 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:47.156+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:48.192 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:48.189+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:49.171 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:49.167+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:50.165 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:50.162+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:51.122 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:51.119+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:52.174 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:52.171+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:52.257 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:55.380 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:52.260 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:55.382 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:52.285 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:29:55.408 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:52.310 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:55.403 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:52.310 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:55.404 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:52.316 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:55.439 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:52.321 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:55.442 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:52.322 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:29:55.419 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:52.338 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:29:55.461 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:53.198 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:53.195+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:54.204 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:54.201+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:55.198 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:55.194+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:56.185 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:56.182+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:57.144 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:57.140+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:57.265 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:00.387 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:57.270 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:00.393 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:57.288 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:00.411 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:57.314 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:00.411 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:57.318 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:00.415 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:57.324 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:00.447 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:57.325 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:00.448 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:57.330 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:00.427 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:57.336 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:00.459 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:21:58.177 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:58.174+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:21:59.153 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:21:59.150+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:00.196 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:00.194+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:01.211 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:01.208+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:02.219 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:02.217+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:03.262 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:03.259+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:04.240 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:04.237+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:05.233 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:05.230+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:06.231 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:06.229+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:07.233 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:07.230+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:07.297 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:10.419 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:07.300 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:10.423 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:07.323 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:10.446 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:07.347 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:10.444 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:07.348 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:10.445 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:07.363 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:10.486 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:07.364 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:10.461 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:07.366 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:10.489 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:07.383 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:10.506 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:08.190 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:08.187+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:09.209 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:09.206+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:10.194 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:10.191+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:11.220 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:11.217+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:12.202 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:12.199+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:13.181 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:13.178+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:14.175 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:14.173+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:15.170 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:15.166+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:16.171 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:16.168+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:17.206 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:17.203+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:17.399 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:20.522 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:17.402 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:20.525 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:17.423 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:20.545 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:17.449 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:20.545 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:17.449 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:20.546 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:17.463 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:20.560 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:17.476 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:20.598 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:17.476 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:20.599 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:17.490 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:20.613 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:18.203 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:18.200+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:19.161 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:19.158+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:20.162 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:20.160+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:21.190 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:21.188+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:22.225 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:22.222+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:22.339 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:25.462 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:22.344 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:25.467 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:22.361 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:25.484 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:22.393 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:25.490 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:22.396 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:25.493 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:22.399 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:25.522 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:22.404 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:25.527 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:22.408 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:25.505 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:22.421 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:25.544 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:23.276 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:23.273+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:24.292 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:24.289+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:25.299 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:25.296+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:26.296 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:26.294+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:27.256 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:27.254+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:28.218 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:28.215+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:29.256 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:29.254+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:30.213 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:30.210+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:30.897 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:22:30.895+0000 10a5e640 -1 osd.0 830 heartbeat_check: no reply from 172.21.15.19:6804 osd.3 since back 2023-10-06T14:21:07.145539+0000 front 2023-10-06T14:21:07.151618+0000 (oldest deadline 2023-10-06T14:22:30.039166+0000) 2023-10-06T14:22:31.198 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:31.195+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:31.902 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:22:31.900+0000 10a5e640 -1 osd.0 830 heartbeat_check: no reply from 172.21.15.19:6804 osd.3 since back 2023-10-06T14:21:07.145539+0000 front 2023-10-06T14:21:07.151618+0000 (oldest deadline 2023-10-06T14:22:30.039166+0000) 2023-10-06T14:22:32.173 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:32.170+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:32.233 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:22:32.231+0000 10a5e640 -1 osd.7 830 heartbeat_check: no reply from 172.21.15.19:6804 osd.3 since back 2023-10-06T14:21:06.020933+0000 front 2023-10-06T14:21:06.017393+0000 (oldest deadline 2023-10-06T14:22:31.904042+0000) 2023-10-06T14:22:32.467 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:35.590 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:32.473 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:35.596 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:32.488 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:35.611 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:32.516 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:35.612 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:32.519 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:35.616 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:32.533 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:35.630 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:32.543 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:35.666 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:32.547 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:35.670 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:32.569 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:35.691 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:32.690 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:22:32.688+0000 1125f640 -1 osd.6 830 heartbeat_check: no reply from 172.21.15.19:6804 osd.3 since back 2023-10-06T14:21:07.364517+0000 front 2023-10-06T14:21:07.369564+0000 (oldest deadline 2023-10-06T14:22:32.059993+0000) 2023-10-06T14:22:32.869 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:22:32.866+0000 10a5e640 -1 osd.0 830 heartbeat_check: no reply from 172.21.15.19:6804 osd.3 since back 2023-10-06T14:21:07.145539+0000 front 2023-10-06T14:21:07.151618+0000 (oldest deadline 2023-10-06T14:22:30.039166+0000) 2023-10-06T14:22:32.951 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:22:32.948+0000 10a5e640 -1 osd.2 830 heartbeat_check: no reply from 172.21.15.19:6804 osd.3 since back 2023-10-06T14:21:06.527555+0000 front 2023-10-06T14:21:06.531771+0000 (oldest deadline 2023-10-06T14:22:32.417930+0000) 2023-10-06T14:22:33.144 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:33.141+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:33.231 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:22:33.228+0000 10a5e640 -1 osd.7 830 heartbeat_check: no reply from 172.21.15.19:6804 osd.3 since back 2023-10-06T14:21:06.020933+0000 front 2023-10-06T14:21:06.017393+0000 (oldest deadline 2023-10-06T14:22:31.904042+0000) 2023-10-06T14:22:33.692 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:22:33.689+0000 1125f640 -1 osd.6 830 heartbeat_check: no reply from 172.21.15.19:6804 osd.3 since back 2023-10-06T14:21:07.364517+0000 front 2023-10-06T14:21:07.369564+0000 (oldest deadline 2023-10-06T14:22:32.059993+0000) 2023-10-06T14:22:33.910 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:22:33.906+0000 10a5e640 -1 osd.2 830 heartbeat_check: no reply from 172.21.15.19:6804 osd.3 since back 2023-10-06T14:21:06.527555+0000 front 2023-10-06T14:21:06.531771+0000 (oldest deadline 2023-10-06T14:22:32.417930+0000) 2023-10-06T14:22:33.925 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:22:33.915+0000 10a5e640 -1 osd.0 830 heartbeat_check: no reply from 172.21.15.19:6804 osd.3 since back 2023-10-06T14:21:07.145539+0000 front 2023-10-06T14:21:07.151618+0000 (oldest deadline 2023-10-06T14:22:30.039166+0000) 2023-10-06T14:22:34.162 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:34.159+0000 1125f640 -1 osd.1 830 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:34.228 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:22:34.226+0000 10a5e640 -1 osd.7 830 heartbeat_check: no reply from 172.21.15.19:6804 osd.3 since back 2023-10-06T14:21:06.020933+0000 front 2023-10-06T14:21:06.017393+0000 (oldest deadline 2023-10-06T14:22:31.904042+0000) 2023-10-06T14:22:35.166 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:22:35.163+0000 1125f640 -1 osd.1 831 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.17105.0:801 244.13 244.28992093 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e829) 2023-10-06T14:22:37.027 INFO:tasks.ceph.osd.3.smithi019.stderr:2023-10-06T14:22:37.024+0000 12261640 -1 osd.3 832 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T14:22:37.731 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:40.854 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:37.734 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:40.857 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:37.755 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:40.878 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:37.774 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:40.870 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:37.775 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:40.872 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:37.790 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:40.887 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:37.806 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:40.928 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:37.816 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:40.939 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:37.833 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:40.955 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:47.513 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:50.636 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:47.515 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:50.638 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:47.538 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:50.661 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:47.560 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:50.657 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:47.561 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:50.658 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:47.569 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:50.692 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:47.571 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:50.694 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:47.586 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:50.683 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:47.598 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:50.721 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:50.339 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:22:50.339 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (10325 ms) 2023-10-06T14:22:50.339 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2023-10-06T14:22:50.339 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:22:50.340 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (7502 ms) 2023-10-06T14:22:50.340 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2023-10-06T14:22:50.340 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:22:50.340 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (6628 ms) 2023-10-06T14:22:50.340 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2023-10-06T14:22:50.340 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:22:50.340 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (26688 ms) 2023-10-06T14:22:50.340 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2023-10-06T14:22:50.340 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:22:50.340 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (15505 ms) 2023-10-06T14:22:50.341 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2023-10-06T14:22:50.341 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (13260 ms) 2023-10-06T14:22:50.341 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2023-10-06T14:22:50.341 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2023-10-06T14:22:50.341 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: 00000010 69 73 |is| 2023-10-06T14:22:50.341 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: 00000012 2023-10-06T14:22:50.341 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (15689 ms) 2023-10-06T14:22:50.341 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2023-10-06T14:22:50.341 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: require_osd_release = reef 2023-10-06T14:22:50.341 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (7766 ms) 2023-10-06T14:22:50.342 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (1300898 ms total) 2023-10-06T14:22:50.342 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: 2023-10-06T14:22:50.342 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2023-10-06T14:22:50.342 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2023-10-06T14:22:50.342 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1323 ms) 2023-10-06T14:22:50.342 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2023-10-06T14:22:50.342 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (12302 ms) 2023-10-06T14:22:50.342 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2023-10-06T14:22:50.343 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (19215 ms) 2023-10-06T14:22:50.343 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2023-10-06T14:22:50.343 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (23 ms) 2023-10-06T14:22:50.343 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (32863 ms total) 2023-10-06T14:22:50.343 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: 2023-10-06T14:22:50.343 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2023-10-06T14:22:50.343 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2023-10-06T14:22:50.343 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (8407 ms) 2023-10-06T14:22:50.343 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2023-10-06T14:22:50.343 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (9769 ms) 2023-10-06T14:22:50.344 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2023-10-06T14:22:50.344 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: waiting for scrub... 2023-10-06T14:22:50.344 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: done waiting 2023-10-06T14:22:50.344 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (26725 ms) 2023-10-06T14:22:50.344 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2023-10-06T14:22:50.344 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (11171 ms) 2023-10-06T14:22:50.344 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2023-10-06T14:22:50.344 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (9063 ms) 2023-10-06T14:22:50.345 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2023-10-06T14:22:50.345 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (67070 ms) 2023-10-06T14:22:50.345 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2023-10-06T14:22:50.345 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (93392 ms) 2023-10-06T14:22:50.345 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2023-10-06T14:22:52.502 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:55.624 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:52.510 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:55.633 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:52.520 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:55.617 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:52.521 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:55.618 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:52.534 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:55.657 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:52.589 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:30:55.686 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:52.604 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:55.727 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:52.630 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:30:55.753 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:52.653 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:30:55.776 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:57.550 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:00.673 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:57.553 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:00.675 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:57.565 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:00.688 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:57.597 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:00.693 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:57.597 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:00.694 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:57.598 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:00.721 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:57.601 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:00.724 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:57.610 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:00.707 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:22:57.615 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:00.738 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:02.562 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:05.684 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:02.566 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:05.689 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:02.582 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:05.704 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:02.612 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:05.709 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:02.613 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:05.710 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:02.618 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:05.741 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:02.624 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:05.719 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:02.624 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:05.744 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:02.631 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:05.754 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:17.608 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:20.731 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:17.614 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:20.737 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:17.655 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:20.778 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:17.660 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:20.757 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:17.663 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:20.760 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:17.686 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:20.808 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:17.689 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:20.812 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:17.694 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:20.791 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:17.719 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:20.842 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:18.323 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:21.420 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:18.324 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:21.421 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:18.333 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:21.429 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:18.647 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:21.770 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:18.649 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:21.771 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:18.665 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:21.788 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:18.666 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:21.788 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:18.668 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:21.791 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:18.683 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:21.806 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:32.653 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:35.776 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:32.654 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:35.777 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:32.680 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:35.802 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:32.695 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:35.792 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:32.697 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:35.794 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:32.700 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:35.823 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:32.704 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:35.827 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:32.722 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:35.819 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:32.729 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:35.852 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:42.888 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:46.011 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:42.891 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:46.013 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:42.927 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:46.050 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:42.942 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:46.038 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:42.947 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:46.044 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:42.948 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:46.070 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:42.960 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:46.082 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:42.977 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:46.074 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:42.996 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:46.119 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:47.767 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:50.889 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:47.770 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:50.893 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:47.788 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:31:50.911 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:47.817 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:50.914 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:47.818 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:50.915 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:47.820 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:50.943 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:47.823 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:50.946 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:47.832 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:31:50.929 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:23:47.838 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:31:50.960 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:02.800 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:05.922 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:02.803 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:05.926 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:02.830 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:05.953 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:02.848 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:05.945 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:02.849 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:05.946 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:02.853 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:05.976 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:02.856 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:05.978 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:02.876 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:05.973 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:02.886 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:06.008 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:03.527 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:06.623 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:03.527 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:06.624 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:03.536 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:06.633 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:03.736 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:06.858 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:03.737 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:06.860 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:03.753 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:06.875 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:03.803 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:06.925 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:03.804 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:06.927 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:03.819 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:06.942 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:17.858 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:20.981 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:17.859 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:20.982 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:17.880 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:21.002 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:17.904 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:21.001 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:17.907 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:21.004 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:17.924 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:21.047 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:17.926 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:21.049 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:17.928 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:21.025 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:17.946 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:21.069 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:22.941 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:26.063 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:22.947 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:26.070 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:22.960 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:26.083 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:22.992 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:26.088 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:22.993 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:26.090 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:22.996 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:26.119 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:22.997 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:26.120 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:23.004 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:26.101 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:23.011 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:26.134 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:32.909 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:36.031 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:32.911 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:36.034 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:32.946 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:36.068 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:32.959 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:36.055 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:32.960 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:36.057 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:32.973 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:36.096 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:32.975 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:36.098 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:32.981 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:36.078 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:32.997 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:36.120 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:43.156 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:46.279 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:43.159 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:46.282 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:43.193 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:46.316 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:43.220 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:46.342 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:43.222 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:46.319 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:43.229 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:46.352 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:43.230 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:46.326 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:43.255 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:46.351 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:43.269 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:46.391 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:48.099 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:51.221 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:48.102 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:51.225 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:48.117 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:32:51.239 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:48.147 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:51.244 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:48.148 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:51.245 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:48.152 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:51.274 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:48.153 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:51.276 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:48.162 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:32:51.258 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:48.169 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:32:51.292 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:59.101 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:02.198 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:59.141 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:02.263 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:59.150 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:02.247 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:59.151 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:02.274 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:59.157 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:02.279 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:59.158 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:02.281 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:59.170 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:02.266 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:59.178 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:02.301 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:24:59.245 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:02.368 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:03.021 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:06.144 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:03.027 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:06.150 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:03.043 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:06.166 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:03.073 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:06.170 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:03.074 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:06.171 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:03.077 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:06.200 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:03.078 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:06.201 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:03.086 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:06.183 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:03.096 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:06.219 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:08.114 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:11.236 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:08.118 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:11.241 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:08.140 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:11.263 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:08.163 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:11.260 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:08.164 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:11.261 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:08.175 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:11.298 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:08.176 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:11.299 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:08.179 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:11.276 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:08.193 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:11.315 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:23.194 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:26.316 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:23.195 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:26.318 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:23.217 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:26.339 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:23.236 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:26.333 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:23.239 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:26.336 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:23.258 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:26.355 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:23.263 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:26.386 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:23.268 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:26.391 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:23.289 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:26.411 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:38.172 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:41.294 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:38.174 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:41.297 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:38.198 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:41.321 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:38.218 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:41.315 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:38.220 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:41.317 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:38.225 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:41.347 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:38.226 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:41.349 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:38.241 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:41.338 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:38.259 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:41.382 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:43.185 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:46.308 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:43.188 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:46.311 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:43.200 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:33:46.322 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:43.236 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:46.333 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:43.237 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:46.334 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:43.243 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:46.365 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:43.244 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:46.367 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:43.248 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:33:46.345 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:43.254 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:33:46.377 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:58.239 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:01.362 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:58.243 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:01.366 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:58.261 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:01.384 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:58.305 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:01.402 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:58.307 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:01.404 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:58.321 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:01.443 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:58.379 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:01.418 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:58.379 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:01.447 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:25:58.379 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:01.468 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:03.274 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:06.396 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:03.276 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:06.399 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:03.287 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:06.409 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:03.321 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:06.443 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:03.322 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:06.419 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:03.324 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:06.420 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:03.326 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:06.449 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:03.334 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:06.430 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:03.337 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:06.459 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:13.351 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:16.473 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:13.354 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:16.477 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:13.380 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:16.503 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:13.410 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:16.506 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:13.414 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:16.511 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:13.419 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:16.542 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:13.426 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:16.548 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:13.436 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:16.533 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:13.452 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:16.574 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:23.460 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:26.583 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:23.461 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:26.584 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:23.484 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:26.606 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:23.511 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:26.607 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:23.512 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:26.609 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:23.524 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:26.646 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:23.526 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:26.649 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:23.535 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:26.632 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:23.553 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:26.675 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:24.587 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (9479 ms) 2023-10-06T14:26:24.587 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2023-10-06T14:26:24.587 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (15042 ms) 2023-10-06T14:26:24.587 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2023-10-06T14:26:24.587 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (10595 ms) 2023-10-06T14:26:24.587 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2023-10-06T14:26:24.587 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (16030 ms) 2023-10-06T14:26:24.587 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2023-10-06T14:26:24.588 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (10059 ms) 2023-10-06T14:26:24.588 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2023-10-06T14:26:24.588 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (9388 ms) 2023-10-06T14:26:24.588 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2023-10-06T14:26:24.588 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: hmm, no HitSet yet 2023-10-06T14:26:24.588 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: ok, hit_set contains 251:602f83fe:::foo:head 2023-10-06T14:26:24.588 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9861 ms) 2023-10-06T14:26:24.588 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2023-10-06T14:26:24.588 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696602250,0 2023-10-06T14:26:24.589 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: first is 1696602250 2023-10-06T14:26:24.589 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696602250,0 2023-10-06T14:26:24.589 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696602250,0 2023-10-06T14:26:24.589 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696602250,1696602252,1696602253,0 2023-10-06T14:26:24.589 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696602250,1696602252,1696602253,0 2023-10-06T14:26:24.589 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696602250,1696602252,1696602253,0 2023-10-06T14:26:24.589 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696602250,1696602252,1696602253,1696602255,1696602256,0 2023-10-06T14:26:24.589 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696602250,1696602252,1696602253,1696602255,1696602256,0 2023-10-06T14:26:24.589 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696602250,1696602252,1696602253,1696602255,1696602256,0 2023-10-06T14:26:24.589 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696602250,1696602252,1696602253,1696602255,1696602256,1696602258,1696602259,0 2023-10-06T14:26:24.590 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696602250,1696602252,1696602253,1696602255,1696602256,1696602258,1696602259,0 2023-10-06T14:26:24.590 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696602250,1696602252,1696602253,1696602255,1696602256,1696602258,1696602259,0 2023-10-06T14:26:24.590 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: got ls 1696602253,1696602255,1696602256,1696602258,1696602259,1696602261,1696602262,0 2023-10-06T14:26:24.590 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: first now 1696602253, trimmed 2023-10-06T14:26:24.590 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (23258 ms) 2023-10-06T14:26:24.590 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2023-10-06T14:26:24.590 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: foo0 2023-10-06T14:26:24.590 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: verifying foo0 is eventually promoted 2023-10-06T14:26:24.590 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (16965 ms) 2023-10-06T14:26:24.591 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2023-10-06T14:26:24.591 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (19864 ms) 2023-10-06T14:26:24.591 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2023-10-06T14:26:24.591 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (23370 ms) 2023-10-06T14:26:24.591 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2023-10-06T14:26:24.591 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (43325 ms) 2023-10-06T14:26:24.591 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2023-10-06T14:26:24.591 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (4722 ms) 2023-10-06T14:26:24.591 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2023-10-06T14:26:24.592 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (4500 ms) 2023-10-06T14:26:24.592 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2023-10-06T14:26:24.592 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (4389 ms) 2023-10-06T14:26:24.592 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (446445 ms total) 2023-10-06T14:26:24.592 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: 2023-10-06T14:26:24.592 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [----------] Global test environment tear-down 2023-10-06T14:26:24.592 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (1930309 ms total) 2023-10-06T14:26:24.592 INFO:tasks.workunit.client.0.smithi019.stdout: api_tier_pp: [ PASSED ] 77 tests. 2023-10-06T14:26:24.596 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.596 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91663 2023-10-06T14:26:24.597 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91663 2023-10-06T14:26:24.597 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.597 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91883 2023-10-06T14:26:24.597 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91883 2023-10-06T14:26:24.597 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.597 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92354 2023-10-06T14:26:24.597 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 92354 2023-10-06T14:26:24.597 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.597 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92111 2023-10-06T14:26:24.597 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 92111 2023-10-06T14:26:24.598 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.598 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91999 2023-10-06T14:26:24.598 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91999 2023-10-06T14:26:24.598 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.598 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91742 2023-10-06T14:26:24.598 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91742 2023-10-06T14:26:24.598 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.598 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92056 2023-10-06T14:26:24.598 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 92056 2023-10-06T14:26:24.599 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.599 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91762 2023-10-06T14:26:24.599 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91762 2023-10-06T14:26:24.599 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.599 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92266 2023-10-06T14:26:24.599 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 92266 2023-10-06T14:26:24.599 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.599 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91637 2023-10-06T14:26:24.599 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91637 2023-10-06T14:26:24.599 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.600 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91698 2023-10-06T14:26:24.600 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91698 2023-10-06T14:26:24.600 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.600 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92288 2023-10-06T14:26:24.600 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 92288 2023-10-06T14:26:24.600 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.600 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91628 2023-10-06T14:26:24.600 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91628 2023-10-06T14:26:24.600 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.600 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91649 2023-10-06T14:26:24.601 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91649 2023-10-06T14:26:24.601 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.601 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91893 2023-10-06T14:26:24.601 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91893 2023-10-06T14:26:24.601 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.601 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92044 2023-10-06T14:26:24.601 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 92044 2023-10-06T14:26:24.601 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.601 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91955 2023-10-06T14:26:24.601 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91955 2023-10-06T14:26:24.602 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.602 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92313 2023-10-06T14:26:24.602 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 92313 2023-10-06T14:26:24.602 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.602 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92198 2023-10-06T14:26:24.602 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 92198 2023-10-06T14:26:24.602 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.602 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91808 2023-10-06T14:26:24.602 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91808 2023-10-06T14:26:24.602 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.603 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92147 2023-10-06T14:26:24.603 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 92147 2023-10-06T14:26:24.603 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.603 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91621 2023-10-06T14:26:24.603 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91621 2023-10-06T14:26:24.603 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.603 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91679 2023-10-06T14:26:24.603 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91679 2023-10-06T14:26:24.604 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.604 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92082 2023-10-06T14:26:24.604 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 92082 2023-10-06T14:26:24.604 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.604 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=91718 2023-10-06T14:26:24.604 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 91718 2023-10-06T14:26:24.604 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.604 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92160 2023-10-06T14:26:24.604 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 92160 2023-10-06T14:26:24.604 INFO:tasks.workunit.client.0.smithi019.stderr:+ for t in "${!pids[@]}" 2023-10-06T14:26:24.605 INFO:tasks.workunit.client.0.smithi019.stderr:+ pid=92179 2023-10-06T14:26:24.605 INFO:tasks.workunit.client.0.smithi019.stderr:+ wait 92179 2023-10-06T14:26:24.605 INFO:tasks.workunit.client.0.smithi019.stderr:+ exit 0 2023-10-06T14:26:24.605 INFO:tasks.workunit.client.0.smithi019.stderr:+ cleanup 2023-10-06T14:26:24.605 INFO:tasks.workunit.client.0.smithi019.stderr:+ pkill -P 91616 2023-10-06T14:26:24.606 INFO:tasks.workunit.client.0.smithi019.stderr:+ true 2023-10-06T14:26:24.607 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-06T14:26:24.607 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-06T14:26:24.640 INFO:tasks.workunit:Stopping ['rados/test.sh'] on client.0... 2023-10-06T14:26:24.640 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-06T14:26:24.977 DEBUG:teuthology.parallel:result is None 2023-10-06T14:26:24.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-10-06T14:26:25.007 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-10-06T14:26:25.007 DEBUG:teuthology.orchestra.run.smithi019:> rmdir -- /home/ubuntu/cephtest/mnt.0 2023-10-06T14:26:25.067 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2023-10-06T14:26:25.067 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T14:26:25.078 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T14:26:25.078 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T14:26:25.213 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.211+0000 7f3eb8af1640 10 monclient: get_monmap_and_config 2023-10-06T14:26:25.213 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.211+0000 7f3eb8af1640 10 monclient: build_initial_monmap 2023-10-06T14:26:25.213 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.211+0000 7f3eb8af1640 10 monclient: monmap: 2023-10-06T14:26:25.213 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:25.213 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:25.214 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:25.212256+0000 2023-10-06T14:26:25.214 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:25.212256+0000 2023-10-06T14:26:25.214 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:25.214 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:25.214 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:25.214 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:25.214 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:25.214 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:25.214 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:25.214 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:25.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.211+0000 7f3eb8af1640 1 Processor -- start 2023-10-06T14:26:25.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.211+0000 7f3eb8af1640 1 -- start start 2023-10-06T14:26:25.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.211+0000 7f3eb8af1640 10 monclient: init 2023-10-06T14:26:25.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.212+0000 7f3eb8af1640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:25.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.212+0000 7f3eb8af1640 10 monclient: _add_conns ranks=[3,2,0,4,1] 2023-10-06T14:26:25.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.212+0000 7f3eb8af1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3eb406b6d0 0x7f3eb4144200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:25.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.212+0000 7f3eb8af1640 10 monclient(hunting): picked mon.noname-a con 0x7f3eb406b6d0 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:25.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.212+0000 7f3eb8af1640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3eb4144740 0x7f3eb413df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:25.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.212+0000 7f3eb8af1640 10 monclient(hunting): picked mon.noname-b con 0x7f3eb4144740 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:25.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.212+0000 7f3eb8af1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3eb413e460 addr v1:172.21.15.19:6789/0 2023-10-06T14:26:25.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.212+0000 7f3eb8af1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb406baa0 con 0x7f3eb413e460 2023-10-06T14:26:25.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.212+0000 7f3eb8af1640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:25.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.212+0000 7f3eb8af1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb40b2530 con 0x7f3eb406b6d0 2023-10-06T14:26:25.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.212+0000 7f3eb8af1640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:25.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.212+0000 7f3eb8af1640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40b2810 con 0x7f3eb4144740 2023-10-06T14:26:25.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.212+0000 7f3eb8af1640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:25.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.212+0000 7f3eb8af1640 10 monclient(hunting): authenticate will time out at 3313.398951s 2023-10-06T14:26:25.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.213+0000 7f3eb1d9b640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3eb4144740 0x7f3eb413df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:25.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.213+0000 7f3eb259c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3eb406b6d0 0x7f3eb4144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:25.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.213+0000 7f3eb1d9b640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3eb4144740 0x7f3eb413df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40174/0 (socket says 172.21.15.19:40174) 2023-10-06T14:26:25.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.213+0000 7f3eb1d9b640 1 -- 172.21.15.19:0/604675400 learned_addr learned my addr 172.21.15.19:0/604675400 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:25.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.213+0000 7f3eb1d9b640 10 monclient(hunting): get_auth_request con 0x7f3eb4144740 auth_method 0 2023-10-06T14:26:25.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.213+0000 7f3eb1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:25.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.213+0000 7f3eb1d9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:25.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.213+0000 7f3eb1d9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:25.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.214+0000 7f3eb259c640 10 monclient(hunting): get_auth_request con 0x7f3eb406b6d0 auth_method 0 2023-10-06T14:26:25.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.214+0000 7f3eb259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:25.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.214+0000 7f3eb259c640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:25.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.214+0000 7f3eb259c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:25.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.214+0000 7f3eb1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:25.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.214+0000 7f3eb1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:25.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.214+0000 7f3eb1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:25.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.214+0000 7f3eb259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:25.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.214+0000 7f3eb259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:25.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.214+0000 7f3eb259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:25.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.216+0000 7f3eb1d9b640 10 monclient(hunting): handle_auth_done global_id 17108 payload 274 2023-10-06T14:26:25.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.216+0000 7f3eb1d9b640 1 -- 172.21.15.19:0/604675400 >> v2:172.21.15.19:3300/0 conn(0x7f3eb406b6d0 msgr2=0x7f3eb4144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:25.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.216+0000 7f3eb1d9b640 1 --2- 172.21.15.19:0/604675400 >> v2:172.21.15.19:3300/0 conn(0x7f3eb406b6d0 0x7f3eb4144200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.216+0000 7f3eb1d9b640 1 -- 172.21.15.19:0/604675400 >> v1:172.21.15.19:6789/0 conn(0x7f3eb413e460 legacy=0x7f3eb40b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:25.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.216+0000 7f3eb1d9b640 10 monclient: _finish_hunting 0 2023-10-06T14:26:25.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.216+0000 7f3eb1d9b640 1 monclient: found mon.noname-b 2023-10-06T14:26:25.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.216+0000 7f3eb1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:25.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.216+0000 7f3eb1d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T14:26:25.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.216+0000 7f3eb1d9b640 1 -- 172.21.15.19:0/604675400 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb40aeff0 con 0x7f3eb4144740 2023-10-06T14:26:25.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.218+0000 7f3eb1d9b640 1 --2- 172.21.15.19:0/604675400 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3eb4144740 0x7f3eb413df20 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009870 tx=0x7f3ea802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce0d3db056688a5e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:25.224 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb159a640 1 -- 172.21.15.19:0/604675400 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8030840 con 0x7f3eb4144740 2023-10-06T14:26:25.224 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:25.224 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:25.224 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb159a640 10 monclient: dump: 2023-10-06T14:26:25.224 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:25.224 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:25.224 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.224 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:25.225 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb159a640 10 monclient: _finish_auth 0 2023-10-06T14:26:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb159a640 10 monclient: _check_auth_tickets 2023-10-06T14:26:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb159a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb159a640 1 -- 172.21.15.19:0/604675400 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea80309a0 con 0x7f3eb4144740 2023-10-06T14:26:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb159a640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb159a640 1 -- 172.21.15.19:0/604675400 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8030d30 con 0x7f3eb4144740 2023-10-06T14:26:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb8af1640 5 monclient: authenticate success, global_id 17108 2023-10-06T14:26:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb8af1640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb8af1640 4 set_mon_vals no callback set 2023-10-06T14:26:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb159a640 10 monclient: dump: 2023-10-06T14:26:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:25.227 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb8af1640 10 monclient: shutdown 2023-10-06T14:26:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb8af1640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb8af1640 1 -- 172.21.15.19:0/604675400 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3eb4144740 msgr2=0x7f3eb413df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb8af1640 1 --2- 172.21.15.19:0/604675400 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3eb4144740 0x7f3eb413df20 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009870 tx=0x7f3ea802b980 comp rx=0 tx=0).stop 2023-10-06T14:26:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb8af1640 1 -- 172.21.15.19:0/604675400 shutdown_connections 2023-10-06T14:26:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb8af1640 1 --2- 172.21.15.19:0/604675400 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3eb4144740 0x7f3eb413df20 unknown :-1 s=CLOSED pgs=3114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb8af1640 1 --2- 172.21.15.19:0/604675400 >> v2:172.21.15.19:3300/0 conn(0x7f3eb406b6d0 0x7f3eb4144200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb8af1640 1 -- 172.21.15.19:0/604675400 >> 172.21.15.19:0/604675400 conn(0x7f3eb40665b0 msgr2=0x7f3eb406ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.222+0000 7f3eb8af1640 1 -- 172.21.15.19:0/604675400 shutdown_connections 2023-10-06T14:26:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.223+0000 7f3eb8af1640 1 -- 172.21.15.19:0/604675400 wait complete. 2023-10-06T14:26:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.223+0000 7f3eb8af1640 10 monclient: build_initial_monmap 2023-10-06T14:26:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.223+0000 7f3eb8af1640 10 monclient: monmap: 2023-10-06T14:26:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:25.232 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 1 librados: starting msgr at 2023-10-06T14:26:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 1 librados: starting objecter 2023-10-06T14:26:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 1 Processor -- start 2023-10-06T14:26:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 1 -- start start 2023-10-06T14:26:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 1 librados: setting wanted keys 2023-10-06T14:26:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 1 librados: calling monclient init 2023-10-06T14:26:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 10 monclient: init 2023-10-06T14:26:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 10 monclient: _add_conns ranks=[2,1,0] 2023-10-06T14:26:25.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3eb406b6d0 0x7f3eb41cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:25.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 10 monclient(hunting): picked mon.noname-c con 0x7f3eb406b6d0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:25.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3eb413e460 0x7f3eb41d62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:25.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 10 monclient(hunting): picked mon.noname-b con 0x7f3eb413e460 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:25.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb4144740 0x7f3eb41d6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:25.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 10 monclient(hunting): picked mon.noname-a con 0x7f3eb4144740 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:25.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:25.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb41d5e60 con 0x7f3eb4144740 2023-10-06T14:26:25.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:25.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb41d6d70 con 0x7f3eb413e460 2023-10-06T14:26:25.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:25.235 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb41ca250 con 0x7f3eb406b6d0 2023-10-06T14:26:25.235 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:25.235 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.224+0000 7f3eb8af1640 10 monclient(hunting): authenticate will time out at 3313.411343s 2023-10-06T14:26:25.235 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.225+0000 7f3eb259c640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3eb406b6d0 0x7f3eb41cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:25.235 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.225+0000 7f3eb2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb4144740 0x7f3eb41d6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:25.235 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.225+0000 7f3eb259c640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3eb406b6d0 0x7f3eb41cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40186/0 (socket says 172.21.15.19:40186) 2023-10-06T14:26:25.235 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.225+0000 7f3eb259c640 1 -- 172.21.15.19:0/3998477296 learned_addr learned my addr 172.21.15.19:0/3998477296 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:25.235 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.225+0000 7f3eb1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3eb413e460 0x7f3eb41d62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:25.235 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.225+0000 7f3eb259c640 10 monclient(hunting): get_auth_request con 0x7f3eb406b6d0 auth_method 0 2023-10-06T14:26:25.235 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.225+0000 7f3eb259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:25.235 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.225+0000 7f3eb259c640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.225+0000 7f3eb259c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.225+0000 7f3eb2d9d640 10 monclient(hunting): get_auth_request con 0x7f3eb4144740 auth_method 0 2023-10-06T14:26:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.225+0000 7f3eb2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.225+0000 7f3eb2d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.225+0000 7f3eb2d9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.226+0000 7f3eb1d9b640 10 monclient(hunting): get_auth_request con 0x7f3eb413e460 auth_method 0 2023-10-06T14:26:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.226+0000 7f3eb1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.226+0000 7f3eb1d9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.226+0000 7f3eb1d9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.226+0000 7f3eb259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.226+0000 7f3eb259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.226+0000 7f3eb259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.226+0000 7f3eb2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.226+0000 7f3eb2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.226+0000 7f3eb2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.226+0000 7f3eb1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.226+0000 7f3eb1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:25.237 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.226+0000 7f3eb1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:25.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.229+0000 7f3eb259c640 10 monclient(hunting): handle_auth_done global_id 17111 payload 824 2023-10-06T14:26:25.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.229+0000 7f3eb259c640 1 -- 172.21.15.19:0/3998477296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3eb413e460 msgr2=0x7f3eb41d62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:25.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.229+0000 7f3eb259c640 1 --2- 172.21.15.19:0/3998477296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3eb413e460 0x7f3eb41d62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.229+0000 7f3eb259c640 1 -- 172.21.15.19:0/3998477296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb4144740 msgr2=0x7f3eb41d6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:25.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.229+0000 7f3eb259c640 1 --2- 172.21.15.19:0/3998477296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb4144740 0x7f3eb41d6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.240 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.229+0000 7f3eb259c640 10 monclient: _finish_hunting 0 2023-10-06T14:26:25.240 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.229+0000 7f3eb259c640 1 monclient: found mon.noname-c 2023-10-06T14:26:25.240 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.229+0000 7f3eb259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:25.240 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.229+0000 7f3eb259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T14:26:25.240 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.229+0000 7f3eb259c640 1 -- 172.21.15.19:0/3998477296 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea80094d0 con 0x7f3eb406b6d0 2023-10-06T14:26:25.240 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.231+0000 7f3eb259c640 1 --2- 172.21.15.19:0/3998477296 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3eb406b6d0 0x7f3eb41cd040 secure :-1 s=READY pgs=3115 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400e9e0 tx=0x7f3ea400eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:25.240 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.231+0000 7f3ea2ffd640 10 client.?.objecter ms_handle_connect 0x7f3eb406b6d0 2023-10-06T14:26:25.240 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.231+0000 7f3ea2ffd640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:25.240 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3ea2ffd640 1 -- 172.21.15.19:0/3998477296 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4009670 con 0x7f3eb406b6d0 2023-10-06T14:26:25.241 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3ea2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:25.241 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3ea2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T14:26:25.241 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3ea2ffd640 10 monclient: dump: 2023-10-06T14:26:25.241 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:25.241 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:25.241 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.241 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.241 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:25.241 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:25.241 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:25.241 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:25.242 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:25.242 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:25.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3ea2ffd640 10 monclient: _finish_auth 0 2023-10-06T14:26:25.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3ea2ffd640 10 monclient: _check_auth_tickets 2023-10-06T14:26:25.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3ea2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:25.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3ea2ffd640 1 -- 172.21.15.19:0/3998477296 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea400c9d0 con 0x7f3eb406b6d0 2023-10-06T14:26:25.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3eb8af1640 5 monclient: authenticate success, global_id 17111 2023-10-06T14:26:25.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3eb8af1640 10 monclient: _renew_subs 2023-10-06T14:26:25.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3eb8af1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:25.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3eb8af1640 1 -- 172.21.15.19:0/3998477296 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb4144c10 con 0x7f3eb406b6d0 2023-10-06T14:26:25.243 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3ea2ffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:25.243 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3ea2ffd640 1 -- 172.21.15.19:0/3998477296 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea400cd80 con 0x7f3eb406b6d0 2023-10-06T14:26:25.243 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3ea2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:25.243 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3ea2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:25.243 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3ea2ffd640 10 monclient: dump: 2023-10-06T14:26:25.243 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:25.243 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:25.243 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.243 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.243 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:25.243 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:25.244 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:25.244 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:25.244 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:25.244 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:25.244 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3eb0d99640 4 set_mon_vals no callback set 2023-10-06T14:26:25.244 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3eb8af1640 10 client.17111.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:25.244 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3eb8af1640 10 monclient: _renew_subs 2023-10-06T14:26:25.244 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3eb8af1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:25.244 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3eb8af1640 1 -- 172.21.15.19:0/3998477296 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb41cd760 con 0x7f3eb406b6d0 2023-10-06T14:26:25.244 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.235+0000 7f3eb8af1640 1 librados: init done 2023-10-06T14:26:25.244 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.236+0000 7f3eb8af1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:25.244 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.236+0000 7f3eb8af1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:25.245 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.236+0000 7f3eb8af1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:25.245 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.236+0000 7f3eb8af1640 1 -- 172.21.15.19:0/3998477296 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb404e000 con 0x7f3eb406b6d0 2023-10-06T14:26:25.256 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.254+0000 7f3ea2ffd640 1 -- 172.21.15.19:0/3998477296 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f3ea4013470 con 0x7f3eb406b6d0 2023-10-06T14:26:25.257 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.254+0000 7f3ea2ffd640 1 --2- 172.21.15.19:0/3998477296 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3e9406c1c0 0x7f3e9406e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:25.257 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.255+0000 7f3eb1d9b640 1 --2- 172.21.15.19:0/3998477296 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3e9406c1c0 0x7f3e9406e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:25.257 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.255+0000 7f3eb1d9b640 10 monclient: get_auth_request con 0x7f3e9406c1c0 auth_method 0 2023-10-06T14:26:25.257 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.255+0000 7f3eb1d9b640 1 --2- 172.21.15.19:0/3998477296 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3e9406c1c0 0x7f3e9406e670 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8036000 tx=0x7f3ea803b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:25.257 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.255+0000 7f3ea2ffd640 10 client.17111.objecter ms_handle_connect 0x7f3e9406c1c0 2023-10-06T14:26:25.258 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.256+0000 7f3ea2ffd640 1 -- 172.21.15.19:0/3998477296 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(969..969 src has 595..969) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f3ea4058990 con 0x7f3eb406b6d0 2023-10-06T14:26:25.258 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.256+0000 7f3ea2ffd640 10 client.17111.objecter ms_dispatch 0x7f3eb4000b90 osd_map(969..969 src has 595..969) v4 2023-10-06T14:26:25.258 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.256+0000 7f3ea2ffd640 3 client.17111.objecter handle_osd_map got epochs [969,969] > 0 2023-10-06T14:26:25.258 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.256+0000 7f3ea2ffd640 3 client.17111.objecter handle_osd_map decoding full epoch 969 2023-10-06T14:26:25.259 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.256+0000 7f3ea2ffd640 20 client.17111.objecter dump_active .. 0 homeless 2023-10-06T14:26:25.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.514+0000 7f3ea2ffd640 1 -- 172.21.15.19:0/3998477296 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f3ea4054cd0 con 0x7f3eb406b6d0 2023-10-06T14:26:25.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.514+0000 7f3ea2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.514+0000 7f3ea2ffd640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:26:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.687+0000 7f3eb8af1640 1 -- 172.21.15.19:0/3998477296 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3eb40af260 con 0x7f3e9406c1c0 2023-10-06T14:26:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.692+0000 7f3ea2ffd640 1 -- 172.21.15.19:0/3998477296 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+151973 (secure 0 0 0) 0x7f3ea803f040 con 0x7f3e9406c1c0 2023-10-06T14:26:25.698 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T14:26:25.700 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2023-10-06T14:26:25.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 10 librados: watch_flush enter 2023-10-06T14:26:25.703 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 10 librados: watch_flush exit 2023-10-06T14:26:25.704 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 20 client.17111.objecter shutdown clearing up homeless session... 2023-10-06T14:26:25.704 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 10 client.17111.objecter successfully canceled tick 2023-10-06T14:26:25.704 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 1 -- 172.21.15.19:0/3998477296 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3e9406c1c0 msgr2=0x7f3e9406e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:25.704 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 1 --2- 172.21.15.19:0/3998477296 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3e9406c1c0 0x7f3e9406e670 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8036000 tx=0x7f3ea803b040 comp rx=0 tx=0).stop 2023-10-06T14:26:25.705 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 10 monclient: shutdown 2023-10-06T14:26:25.706 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:25.706 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 1 -- 172.21.15.19:0/3998477296 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3eb406b6d0 msgr2=0x7f3eb41cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:25.706 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 1 --2- 172.21.15.19:0/3998477296 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3eb406b6d0 0x7f3eb41cd040 secure :-1 s=READY pgs=3115 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400e9e0 tx=0x7f3ea400eea0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.706 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 1 -- 172.21.15.19:0/3998477296 shutdown_connections 2023-10-06T14:26:25.706 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 1 --2- 172.21.15.19:0/3998477296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb4144740 0x7f3eb41d6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.706 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 1 --2- 172.21.15.19:0/3998477296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3eb413e460 0x7f3eb41d62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.706 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 1 --2- 172.21.15.19:0/3998477296 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3e9406c1c0 0x7f3e9406e670 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.706 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 1 --2- 172.21.15.19:0/3998477296 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3eb406b6d0 0x7f3eb41cd040 unknown :-1 s=CLOSED pgs=3115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.706 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 1 -- 172.21.15.19:0/3998477296 >> 172.21.15.19:0/3998477296 conn(0x7f3eb40665b0 msgr2=0x7f3eb40b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:25.707 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 1 -- 172.21.15.19:0/3998477296 shutdown_connections 2023-10-06T14:26:25.707 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 1 -- 172.21.15.19:0/3998477296 wait complete. 2023-10-06T14:26:25.707 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.695+0000 7f3eb8af1640 1 librados: shutdown 2023-10-06T14:26:25.709 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":1971,"stamp":"2023-10-06T14:26:24.559354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":496780,"kb_used_data":45004,"kb_used_omap":0,"kb_used_meta":451712,"kb_avail":754477940,"statfs":{"total":773094113280,"available":772585410560,"internally_reserved":0,"allocated":46084096,"data_stored":25057027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":462553088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":417,"apply_latency_ms":417,"commit_latency_ns":417000000,"apply_latency_ns":417000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4000309,"num_objects":-45,"num_object_clones":-1,"num_object_copies":-135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-45,"num_whiteouts":0,"num_read":-30,"num_read_kb":-10,"num_write":-70,"num_write_kb":-3969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.902021"},"pg_stats":[{"pgid":"8.14","version":"0'0","reported_seq":917,"reported_epoch":962,"state":"active+clean","last_fresh":"2023-10-06T14:26:17.564169+0000","last_change":"2023-10-06T13:54:27.674098+0000","last_active":"2023-10-06T14:26:17.564169+0000","last_peered":"2023-10-06T14:26:17.564169+0000","last_clean":"2023-10-06T14:26:17.564169+0000","last_became_active":"2023-10-06T13:54:27.664679+0000","last_became_peered":"2023-10-06T13:54:27.664679+0000","last_unstale":"2023-10-06T14:26:17.564169+0000","last_undegraded":"2023-10-06T14:26:17.564169+0000","last_fullsized":"2023-10-06T14:26:17.564169+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:57:24.020996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":920,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.836623+0000","last_change":"2023-10-06T14:22:39.751625+0000","last_active":"2023-10-06T14:26:22.836623+0000","last_peered":"2023-10-06T14:26:22.836623+0000","last_clean":"2023-10-06T14:26:22.836623+0000","last_became_active":"2023-10-06T14:22:39.724017+0000","last_became_peered":"2023-10-06T14:22:39.724017+0000","last_unstale":"2023-10-06T14:26:22.836623+0000","last_undegraded":"2023-10-06T14:26:22.836623+0000","last_fullsized":"2023-10-06T14:26:22.836623+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:31.357267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":924,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.734022+0000","last_change":"2023-10-06T14:16:33.556535+0000","last_active":"2023-10-06T14:26:20.734022+0000","last_peered":"2023-10-06T14:26:20.734022+0000","last_clean":"2023-10-06T14:26:20.734022+0000","last_became_active":"2023-10-06T14:16:33.542923+0000","last_became_peered":"2023-10-06T14:16:33.542923+0000","last_unstale":"2023-10-06T14:26:20.734022+0000","last_undegraded":"2023-10-06T14:26:20.734022+0000","last_fullsized":"2023-10-06T14:26:20.734022+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:56:19.999321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":922,"reported_epoch":962,"state":"active+clean","last_fresh":"2023-10-06T14:26:17.425528+0000","last_change":"2023-10-06T13:54:29.208698+0000","last_active":"2023-10-06T14:26:17.425528+0000","last_peered":"2023-10-06T14:26:17.425528+0000","last_clean":"2023-10-06T14:26:17.425528+0000","last_became_active":"2023-10-06T13:54:29.163161+0000","last_became_peered":"2023-10-06T13:54:29.163161+0000","last_unstale":"2023-10-06T14:26:17.425528+0000","last_undegraded":"2023-10-06T14:26:17.425528+0000","last_fullsized":"2023-10-06T14:26:17.425528+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:22.518227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":919,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.795205+0000","last_change":"2023-10-06T14:22:39.220927+0000","last_active":"2023-10-06T14:26:22.795205+0000","last_peered":"2023-10-06T14:26:22.795205+0000","last_clean":"2023-10-06T14:26:22.795205+0000","last_became_active":"2023-10-06T14:22:39.128927+0000","last_became_peered":"2023-10-06T14:22:39.128927+0000","last_unstale":"2023-10-06T14:26:22.795205+0000","last_undegraded":"2023-10-06T14:26:22.795205+0000","last_fullsized":"2023-10-06T14:26:22.795205+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:02.237203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":920,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.885756+0000","last_change":"2023-10-06T14:22:39.468045+0000","last_active":"2023-10-06T14:26:22.885756+0000","last_peered":"2023-10-06T14:26:22.885756+0000","last_clean":"2023-10-06T14:26:22.885756+0000","last_became_active":"2023-10-06T14:22:39.418221+0000","last_became_peered":"2023-10-06T14:22:39.418221+0000","last_unstale":"2023-10-06T14:26:22.885756+0000","last_undegraded":"2023-10-06T14:26:22.885756+0000","last_fullsized":"2023-10-06T14:26:22.885756+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:46.111461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":920,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.866951+0000","last_change":"2023-10-06T14:22:39.210447+0000","last_active":"2023-10-06T14:26:22.866951+0000","last_peered":"2023-10-06T14:26:22.866951+0000","last_clean":"2023-10-06T14:26:22.866951+0000","last_became_active":"2023-10-06T14:22:39.195040+0000","last_became_peered":"2023-10-06T14:22:39.195040+0000","last_unstale":"2023-10-06T14:26:22.866951+0000","last_undegraded":"2023-10-06T14:26:22.866951+0000","last_fullsized":"2023-10-06T14:26:22.866951+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:29:47.926237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":925,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.729122+0000","last_change":"2023-10-06T14:16:33.305137+0000","last_active":"2023-10-06T14:26:20.729122+0000","last_peered":"2023-10-06T14:26:20.729122+0000","last_clean":"2023-10-06T14:26:20.729122+0000","last_became_active":"2023-10-06T14:16:33.300222+0000","last_became_peered":"2023-10-06T14:16:33.300222+0000","last_unstale":"2023-10-06T14:26:20.729122+0000","last_undegraded":"2023-10-06T14:26:20.729122+0000","last_fullsized":"2023-10-06T14:26:20.729122+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:34:52.750381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":919,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.822743+0000","last_change":"2023-10-06T14:22:39.745594+0000","last_active":"2023-10-06T14:26:22.822743+0000","last_peered":"2023-10-06T14:26:22.822743+0000","last_clean":"2023-10-06T14:26:22.822743+0000","last_became_active":"2023-10-06T14:22:39.712413+0000","last_became_peered":"2023-10-06T14:22:39.712413+0000","last_unstale":"2023-10-06T14:26:22.822743+0000","last_undegraded":"2023-10-06T14:26:22.822743+0000","last_fullsized":"2023-10-06T14:26:22.822743+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:59:53.774197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":905,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.206280+0000","last_change":"2023-10-06T13:54:19.475064+0000","last_active":"2023-10-06T14:26:21.206280+0000","last_peered":"2023-10-06T14:26:21.206280+0000","last_clean":"2023-10-06T14:26:21.206280+0000","last_became_active":"2023-10-06T13:54:19.427005+0000","last_became_peered":"2023-10-06T13:54:19.427005+0000","last_unstale":"2023-10-06T14:26:21.206280+0000","last_undegraded":"2023-10-06T14:26:21.206280+0000","last_fullsized":"2023-10-06T14:26:21.206280+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:21.985890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":905,"reported_epoch":963,"state":"active+clean","last_fresh":"2023-10-06T14:26:18.663174+0000","last_change":"2023-10-06T13:54:19.536231+0000","last_active":"2023-10-06T14:26:18.663174+0000","last_peered":"2023-10-06T14:26:18.663174+0000","last_clean":"2023-10-06T14:26:18.663174+0000","last_became_active":"2023-10-06T13:54:19.531077+0000","last_became_peered":"2023-10-06T13:54:19.531077+0000","last_unstale":"2023-10-06T14:26:18.663174+0000","last_undegraded":"2023-10-06T14:26:18.663174+0000","last_fullsized":"2023-10-06T14:26:18.663174+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:05.864633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":917,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.607069+0000","last_change":"2023-10-06T14:06:28.635986+0000","last_active":"2023-10-06T14:26:19.607069+0000","last_peered":"2023-10-06T14:26:19.607069+0000","last_clean":"2023-10-06T14:26:19.607069+0000","last_became_active":"2023-10-06T14:06:28.630618+0000","last_became_peered":"2023-10-06T14:06:28.630618+0000","last_unstale":"2023-10-06T14:26:19.607069+0000","last_undegraded":"2023-10-06T14:26:19.607069+0000","last_fullsized":"2023-10-06T14:26:19.607069+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:32.474746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":927,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.694758+0000","last_change":"2023-10-06T13:54:21.830131+0000","last_active":"2023-10-06T14:26:19.694758+0000","last_peered":"2023-10-06T14:26:19.694758+0000","last_clean":"2023-10-06T14:26:19.694758+0000","last_became_active":"2023-10-06T13:54:21.813915+0000","last_became_peered":"2023-10-06T13:54:21.813915+0000","last_unstale":"2023-10-06T14:26:19.694758+0000","last_undegraded":"2023-10-06T14:26:19.694758+0000","last_fullsized":"2023-10-06T14:26:19.694758+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:21:55.469659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":922,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.832923+0000","last_change":"2023-10-06T14:22:39.679043+0000","last_active":"2023-10-06T14:26:22.832923+0000","last_peered":"2023-10-06T14:26:22.832923+0000","last_clean":"2023-10-06T14:26:22.832923+0000","last_became_active":"2023-10-06T14:22:39.673285+0000","last_became_peered":"2023-10-06T14:22:39.673285+0000","last_unstale":"2023-10-06T14:26:22.832923+0000","last_undegraded":"2023-10-06T14:26:22.832923+0000","last_fullsized":"2023-10-06T14:26:22.832923+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:12.259360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":913,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.740540+0000","last_change":"2023-10-06T14:16:33.403287+0000","last_active":"2023-10-06T14:26:20.740540+0000","last_peered":"2023-10-06T14:26:20.740540+0000","last_clean":"2023-10-06T14:26:20.740540+0000","last_became_active":"2023-10-06T14:16:33.392050+0000","last_became_peered":"2023-10-06T14:16:33.392050+0000","last_unstale":"2023-10-06T14:26:20.740540+0000","last_undegraded":"2023-10-06T14:26:20.740540+0000","last_fullsized":"2023-10-06T14:26:20.740540+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:29:10.779601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":920,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.617510+0000","last_change":"2023-10-06T14:05:13.253436+0000","last_active":"2023-10-06T14:26:19.617510+0000","last_peered":"2023-10-06T14:26:19.617510+0000","last_clean":"2023-10-06T14:26:19.617510+0000","last_became_active":"2023-10-06T14:05:13.248166+0000","last_became_peered":"2023-10-06T14:05:13.248166+0000","last_unstale":"2023-10-06T14:26:19.617510+0000","last_undegraded":"2023-10-06T14:26:19.617510+0000","last_fullsized":"2023-10-06T14:26:19.617510+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:51:29.023023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":903,"reported_epoch":963,"state":"active+clean","last_fresh":"2023-10-06T14:26:18.672148+0000","last_change":"2023-10-06T13:54:26.224788+0000","last_active":"2023-10-06T14:26:18.672148+0000","last_peered":"2023-10-06T14:26:18.672148+0000","last_clean":"2023-10-06T14:26:18.672148+0000","last_became_active":"2023-10-06T13:54:26.218851+0000","last_became_peered":"2023-10-06T13:54:26.218851+0000","last_unstale":"2023-10-06T14:26:18.672148+0000","last_undegraded":"2023-10-06T14:26:18.672148+0000","last_fullsized":"2023-10-06T14:26:18.672148+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:43:20.812836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":939,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.838339+0000","last_change":"2023-10-06T14:22:39.234795+0000","last_active":"2023-10-06T14:26:22.838339+0000","last_peered":"2023-10-06T14:26:22.838339+0000","last_clean":"2023-10-06T14:26:22.838339+0000","last_became_active":"2023-10-06T14:22:39.109893+0000","last_became_peered":"2023-10-06T14:22:39.109893+0000","last_unstale":"2023-10-06T14:26:22.838339+0000","last_undegraded":"2023-10-06T14:26:22.838339+0000","last_fullsized":"2023-10-06T14:26:22.838339+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:27.039844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":925,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.735476+0000","last_change":"2023-10-06T14:16:33.559587+0000","last_active":"2023-10-06T14:26:20.735476+0000","last_peered":"2023-10-06T14:26:20.735476+0000","last_clean":"2023-10-06T14:26:20.735476+0000","last_became_active":"2023-10-06T14:16:33.551586+0000","last_became_peered":"2023-10-06T14:16:33.551586+0000","last_unstale":"2023-10-06T14:26:20.735476+0000","last_undegraded":"2023-10-06T14:26:20.735476+0000","last_fullsized":"2023-10-06T14:26:20.735476+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:53.251968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":918,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.741987+0000","last_change":"2023-10-06T14:16:33.406336+0000","last_active":"2023-10-06T14:26:20.741987+0000","last_peered":"2023-10-06T14:26:20.741987+0000","last_clean":"2023-10-06T14:26:20.741987+0000","last_became_active":"2023-10-06T14:16:33.398349+0000","last_became_peered":"2023-10-06T14:16:33.398349+0000","last_unstale":"2023-10-06T14:26:20.741987+0000","last_undegraded":"2023-10-06T14:26:20.741987+0000","last_fullsized":"2023-10-06T14:26:20.741987+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:24:58.230306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":923,"reported_epoch":962,"state":"active+clean","last_fresh":"2023-10-06T14:26:17.325893+0000","last_change":"2023-10-06T13:54:31.926742+0000","last_active":"2023-10-06T14:26:17.325893+0000","last_peered":"2023-10-06T14:26:17.325893+0000","last_clean":"2023-10-06T14:26:17.325893+0000","last_became_active":"2023-10-06T13:54:31.908571+0000","last_became_peered":"2023-10-06T13:54:31.908571+0000","last_unstale":"2023-10-06T14:26:17.325893+0000","last_undegraded":"2023-10-06T14:26:17.325893+0000","last_fullsized":"2023-10-06T14:26:17.325893+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:45.581588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":928,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.868651+0000","last_change":"2023-10-06T14:22:39.277750+0000","last_active":"2023-10-06T14:26:22.868651+0000","last_peered":"2023-10-06T14:26:22.868651+0000","last_clean":"2023-10-06T14:26:22.868651+0000","last_became_active":"2023-10-06T14:22:39.078831+0000","last_became_peered":"2023-10-06T14:22:39.078831+0000","last_unstale":"2023-10-06T14:26:22.868651+0000","last_undegraded":"2023-10-06T14:26:22.868651+0000","last_fullsized":"2023-10-06T14:26:22.868651+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:52.651981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":916,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.595315+0000","last_change":"2023-10-06T14:05:13.235606+0000","last_active":"2023-10-06T14:26:19.595315+0000","last_peered":"2023-10-06T14:26:19.595315+0000","last_clean":"2023-10-06T14:26:19.595315+0000","last_became_active":"2023-10-06T14:05:13.230098+0000","last_became_peered":"2023-10-06T14:05:13.230098+0000","last_unstale":"2023-10-06T14:26:19.595315+0000","last_undegraded":"2023-10-06T14:26:19.595315+0000","last_fullsized":"2023-10-06T14:26:19.595315+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:28:31.276617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":923,"reported_epoch":963,"state":"active+clean","last_fresh":"2023-10-06T14:26:18.624719+0000","last_change":"2023-10-06T14:16:33.641985+0000","last_active":"2023-10-06T14:26:18.624719+0000","last_peered":"2023-10-06T14:26:18.624719+0000","last_clean":"2023-10-06T14:26:18.624719+0000","last_became_active":"2023-10-06T14:16:33.635681+0000","last_became_peered":"2023-10-06T14:16:33.635681+0000","last_unstale":"2023-10-06T14:26:18.624719+0000","last_undegraded":"2023-10-06T14:26:18.624719+0000","last_fullsized":"2023-10-06T14:26:18.624719+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:42.455510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":927,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.720626+0000","last_change":"2023-10-06T14:16:33.275781+0000","last_active":"2023-10-06T14:26:20.720626+0000","last_peered":"2023-10-06T14:26:20.720626+0000","last_clean":"2023-10-06T14:26:20.720626+0000","last_became_active":"2023-10-06T14:16:33.269648+0000","last_became_peered":"2023-10-06T14:16:33.269648+0000","last_unstale":"2023-10-06T14:26:20.720626+0000","last_undegraded":"2023-10-06T14:26:20.720626+0000","last_fullsized":"2023-10-06T14:26:20.720626+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:25.574583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":938,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.797168+0000","last_change":"2023-10-06T14:22:39.170180+0000","last_active":"2023-10-06T14:26:22.797168+0000","last_peered":"2023-10-06T14:26:22.797168+0000","last_clean":"2023-10-06T14:26:22.797168+0000","last_became_active":"2023-10-06T14:22:39.121176+0000","last_became_peered":"2023-10-06T14:22:39.121176+0000","last_unstale":"2023-10-06T14:26:22.797168+0000","last_undegraded":"2023-10-06T14:26:22.797168+0000","last_fullsized":"2023-10-06T14:26:22.797168+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:24.770331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":918,"reported_epoch":962,"state":"active+clean","last_fresh":"2023-10-06T14:26:17.539905+0000","last_change":"2023-10-06T13:54:27.649937+0000","last_active":"2023-10-06T14:26:17.539905+0000","last_peered":"2023-10-06T14:26:17.539905+0000","last_clean":"2023-10-06T14:26:17.539905+0000","last_became_active":"2023-10-06T13:54:27.642873+0000","last_became_peered":"2023-10-06T13:54:27.642873+0000","last_unstale":"2023-10-06T14:26:17.539905+0000","last_undegraded":"2023-10-06T14:26:17.539905+0000","last_fullsized":"2023-10-06T14:26:17.539905+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:36:36.269635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":926,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.805049+0000","last_change":"2023-10-06T14:22:39.756236+0000","last_active":"2023-10-06T14:26:22.805049+0000","last_peered":"2023-10-06T14:26:22.805049+0000","last_clean":"2023-10-06T14:26:22.805049+0000","last_became_active":"2023-10-06T14:22:39.733473+0000","last_became_peered":"2023-10-06T14:22:39.733473+0000","last_unstale":"2023-10-06T14:26:22.805049+0000","last_undegraded":"2023-10-06T14:26:22.805049+0000","last_fullsized":"2023-10-06T14:26:22.805049+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:09.280243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":1048,"reported_epoch":962,"state":"active+clean","last_fresh":"2023-10-06T14:26:17.327801+0000","last_change":"2023-10-06T14:16:33.639507+0000","last_active":"2023-10-06T14:26:17.327801+0000","last_peered":"2023-10-06T14:26:17.327801+0000","last_clean":"2023-10-06T14:26:17.327801+0000","last_became_active":"2023-10-06T14:16:33.632776+0000","last_became_peered":"2023-10-06T14:16:33.632776+0000","last_unstale":"2023-10-06T14:26:17.327801+0000","last_undegraded":"2023-10-06T14:26:17.327801+0000","last_fullsized":"2023-10-06T14:26:17.327801+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:08.156870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:08.156870+0000","last_clean_scrub_stamp":"2023-10-06T13:53:08.156870+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:08.683941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":903,"reported_epoch":963,"state":"active+clean","last_fresh":"2023-10-06T14:26:18.639457+0000","last_change":"2023-10-06T13:54:26.199772+0000","last_active":"2023-10-06T14:26:18.639457+0000","last_peered":"2023-10-06T14:26:18.639457+0000","last_clean":"2023-10-06T14:26:18.639457+0000","last_became_active":"2023-10-06T13:54:26.193354+0000","last_became_peered":"2023-10-06T13:54:26.193354+0000","last_unstale":"2023-10-06T14:26:18.639457+0000","last_undegraded":"2023-10-06T14:26:18.639457+0000","last_fullsized":"2023-10-06T14:26:18.639457+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:11.930838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":928,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.609432+0000","last_change":"2023-10-06T13:54:22.015064+0000","last_active":"2023-10-06T14:26:19.609432+0000","last_peered":"2023-10-06T14:26:19.609432+0000","last_clean":"2023-10-06T14:26:19.609432+0000","last_became_active":"2023-10-06T13:54:22.005243+0000","last_became_peered":"2023-10-06T13:54:22.005243+0000","last_unstale":"2023-10-06T14:26:19.609432+0000","last_undegraded":"2023-10-06T14:26:19.609432+0000","last_fullsized":"2023-10-06T14:26:19.609432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:46:20.602620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"43'7","reported_seq":935,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.193835+0000","last_change":"2023-10-06T14:05:13.464997+0000","last_active":"2023-10-06T14:26:21.193835+0000","last_peered":"2023-10-06T14:26:21.193835+0000","last_clean":"2023-10-06T14:26:21.193835+0000","last_became_active":"2023-10-06T14:05:13.458219+0000","last_became_peered":"2023-10-06T14:05:13.458219+0000","last_unstale":"2023-10-06T14:26:21.193835+0000","last_undegraded":"2023-10-06T14:26:21.193835+0000","last_fullsized":"2023-10-06T14:26:21.193835+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:40:26.823980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":928,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.694527+0000","last_change":"2023-10-06T14:22:39.278351+0000","last_active":"2023-10-06T14:26:20.694527+0000","last_peered":"2023-10-06T14:26:20.694527+0000","last_clean":"2023-10-06T14:26:20.694527+0000","last_became_active":"2023-10-06T14:22:39.269435+0000","last_became_peered":"2023-10-06T14:22:39.269435+0000","last_unstale":"2023-10-06T14:26:20.694527+0000","last_undegraded":"2023-10-06T14:26:20.694527+0000","last_fullsized":"2023-10-06T14:26:20.694527+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:10.817190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":916,"reported_epoch":963,"state":"active+clean","last_fresh":"2023-10-06T14:26:18.673683+0000","last_change":"2023-10-06T14:16:33.659581+0000","last_active":"2023-10-06T14:26:18.673683+0000","last_peered":"2023-10-06T14:26:18.673683+0000","last_clean":"2023-10-06T14:26:18.673683+0000","last_became_active":"2023-10-06T14:16:33.650127+0000","last_became_peered":"2023-10-06T14:16:33.650127+0000","last_unstale":"2023-10-06T14:26:18.673683+0000","last_undegraded":"2023-10-06T14:26:18.673683+0000","last_fullsized":"2023-10-06T14:26:18.673683+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:45.333800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":907,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.213411+0000","last_change":"2023-10-06T13:54:22.165757+0000","last_active":"2023-10-06T14:26:21.213411+0000","last_peered":"2023-10-06T14:26:21.213411+0000","last_clean":"2023-10-06T14:26:21.213411+0000","last_became_active":"2023-10-06T13:54:22.160421+0000","last_became_peered":"2023-10-06T13:54:22.160421+0000","last_unstale":"2023-10-06T14:26:21.213411+0000","last_undegraded":"2023-10-06T14:26:21.213411+0000","last_fullsized":"2023-10-06T14:26:21.213411+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:51.552552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":924,"reported_epoch":962,"state":"active+clean","last_fresh":"2023-10-06T14:26:17.387295+0000","last_change":"2023-10-06T13:54:25.399248+0000","last_active":"2023-10-06T14:26:17.387295+0000","last_peered":"2023-10-06T14:26:17.387295+0000","last_clean":"2023-10-06T14:26:17.387295+0000","last_became_active":"2023-10-06T13:54:25.349894+0000","last_became_peered":"2023-10-06T13:54:25.349894+0000","last_unstale":"2023-10-06T14:26:17.387295+0000","last_undegraded":"2023-10-06T14:26:17.387295+0000","last_fullsized":"2023-10-06T14:26:17.387295+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:12:02.751197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":941,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.684113+0000","last_change":"2023-10-06T14:22:39.984989+0000","last_active":"2023-10-06T14:26:19.684113+0000","last_peered":"2023-10-06T14:26:19.684113+0000","last_clean":"2023-10-06T14:26:19.684113+0000","last_became_active":"2023-10-06T14:22:39.979167+0000","last_became_peered":"2023-10-06T14:22:39.979167+0000","last_unstale":"2023-10-06T14:26:19.684113+0000","last_undegraded":"2023-10-06T14:26:19.684113+0000","last_fullsized":"2023-10-06T14:26:19.684113+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":836,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:55:13.743000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":937,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.635697+0000","last_change":"2023-10-06T14:05:13.200912+0000","last_active":"2023-10-06T14:26:19.635697+0000","last_peered":"2023-10-06T14:26:19.635697+0000","last_clean":"2023-10-06T14:26:19.635697+0000","last_became_active":"2023-10-06T14:05:13.191757+0000","last_became_peered":"2023-10-06T14:05:13.191757+0000","last_unstale":"2023-10-06T14:26:19.635697+0000","last_undegraded":"2023-10-06T14:26:19.635697+0000","last_fullsized":"2023-10-06T14:26:19.635697+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:21:31.319110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":906,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.187963+0000","last_change":"2023-10-06T13:54:22.153461+0000","last_active":"2023-10-06T14:26:21.187963+0000","last_peered":"2023-10-06T14:26:21.187963+0000","last_clean":"2023-10-06T14:26:21.187963+0000","last_became_active":"2023-10-06T13:54:22.144576+0000","last_became_peered":"2023-10-06T13:54:22.144576+0000","last_unstale":"2023-10-06T14:26:21.187963+0000","last_undegraded":"2023-10-06T14:26:21.187963+0000","last_fullsized":"2023-10-06T14:26:21.187963+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:55:12.616531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":922,"reported_epoch":962,"state":"active+clean","last_fresh":"2023-10-06T14:26:17.330943+0000","last_change":"2023-10-06T13:54:19.740406+0000","last_active":"2023-10-06T14:26:17.330943+0000","last_peered":"2023-10-06T14:26:17.330943+0000","last_clean":"2023-10-06T14:26:17.330943+0000","last_became_active":"2023-10-06T13:54:19.734381+0000","last_became_peered":"2023-10-06T13:54:19.734381+0000","last_unstale":"2023-10-06T14:26:17.330943+0000","last_undegraded":"2023-10-06T14:26:17.330943+0000","last_fullsized":"2023-10-06T14:26:17.330943+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:23.226096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":930,"reported_epoch":962,"state":"active+clean","last_fresh":"2023-10-06T14:26:17.506472+0000","last_change":"2023-10-06T14:05:13.451797+0000","last_active":"2023-10-06T14:26:17.506472+0000","last_peered":"2023-10-06T14:26:17.506472+0000","last_clean":"2023-10-06T14:26:17.506472+0000","last_became_active":"2023-10-06T14:05:13.442612+0000","last_became_peered":"2023-10-06T14:05:13.442612+0000","last_unstale":"2023-10-06T14:26:17.506472+0000","last_undegraded":"2023-10-06T14:26:17.506472+0000","last_fullsized":"2023-10-06T14:26:17.506472+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:59:33.582696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719477121,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59092,"kb_used_data":5900,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312748,"statfs":{"total":96636764160,"available":96576253952,"internally_reserved":0,"allocated":6041600,"data_stored":3419982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":11.946,"5min":12.488,"15min":13.279},"min":{"1min":6.3920000000000003,"5min":4.6070000000000002,"15min":4.6070000000000002},"max":{"1min":52.079999999999998,"5min":91.744,"15min":109.691},"last":9.3190000000000008},{"interface":"front","average":{"1min":12.875999999999999,"5min":12.664,"15min":14.576000000000001},"min":{"1min":4.1870000000000003,"5min":4.1870000000000003,"15min":4.1870000000000003},"max":{"1min":44.232999999999997,"5min":67.936000000000007,"15min":166.33699999999999},"last":11.087999999999999}]},{"osd":2,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":12.112,"5min":11.595000000000001,"15min":13.598000000000001},"min":{"1min":7.9850000000000003,"5min":6.3920000000000003,"15min":4.9800000000000004},"max":{"1min":18.631,"5min":64.661000000000001,"15min":155.68799999999999},"last":18.631},{"interface":"front","average":{"1min":15.757999999999999,"5min":13.82,"15min":16.335000000000001},"min":{"1min":9.0660000000000007,"5min":7.3049999999999997,"15min":6.6059999999999999},"max":{"1min":47.368000000000002,"5min":92.335999999999999,"15min":212.16399999999999},"last":17.513000000000002}]},{"osd":3,"last update":"Fri Oct 6 14:25:44 2023","interfaces":[{"interface":"back","average":{"1min":23.273,"5min":29.661000000000001,"15min":33.582999999999998},"min":{"1min":9.6639999999999997,"5min":8.6850000000000005,"15min":8.6850000000000005},"max":{"1min":126.22799999999999,"5min":310.19,"15min":310.19},"last":9.6609999999999996},{"interface":"front","average":{"1min":18.346,"5min":18.327999999999999,"15min":18.873999999999999},"min":{"1min":8.5370000000000008,"5min":7.8399999999999999,"15min":7.8399999999999999},"max":{"1min":65.375,"5min":65.375,"15min":65.375},"last":13.967000000000001}]},{"osd":4,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":15290.522000000001,"5min":3070.721,"15min":1038.3879999999999},"min":{"1min":9.2720000000000002,"5min":7.8520000000000003,"15min":7.0190000000000001},"max":{"1min":39693.332999999999,"5min":39693.332999999999,"15min":39693.332999999999},"last":12.352},{"interface":"front","average":{"1min":15323.151,"5min":3078.1880000000001,"15min":1039.8889999999999},"min":{"1min":14.579000000000001,"5min":8.5679999999999996,"15min":7.3239999999999998},"max":{"1min":39697.402000000002,"5min":39697.402000000002,"15min":39697.402000000002},"last":14.579000000000001}]},{"osd":5,"last update":"Fri Oct 6 14:25:44 2023","interfaces":[{"interface":"back","average":{"1min":21.614999999999998,"5min":17.170000000000002,"15min":92.123000000000005},"min":{"1min":9.1199999999999992,"5min":9.1199999999999992,"15min":9.1199999999999992},"max":{"1min":69.561999999999998,"5min":69.561999999999998,"15min":8890.4069999999992},"last":10.494},{"interface":"front","average":{"1min":25.356000000000002,"5min":17.224,"15min":92.638999999999996},"min":{"1min":8.3140000000000001,"5min":7.8620000000000001,"15min":7.8620000000000001},"max":{"1min":139.78800000000001,"5min":139.78800000000001,"15min":8888.7939999999999},"last":13.137}]},{"osd":6,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.563000000000001,"5min":15.667999999999999,"15min":1233.97},"min":{"1min":9.9359999999999999,"5min":8.4710000000000001,"15min":7.3529999999999998},"max":{"1min":20.318000000000001,"5min":78.221999999999994,"15min":47304.226999999999},"last":15.303000000000001},{"interface":"front","average":{"1min":17.091000000000001,"5min":18.989000000000001,"15min":1237.328},"min":{"1min":11.202999999999999,"5min":10.666,"15min":8.9949999999999992},"max":{"1min":24.579000000000001,"5min":116.84999999999999,"15min":47302.955999999998},"last":17.899999999999999}]},{"osd":7,"last update":"Fri Oct 6 14:25:31 2023","interfaces":[{"interface":"back","average":{"1min":26.558,"5min":20.433,"15min":22.890000000000001},"min":{"1min":14.305999999999999,"5min":10.992000000000001,"15min":9.0250000000000004},"max":{"1min":78.769000000000005,"5min":78.769000000000005,"15min":214.02699999999999},"last":23.812000000000001},{"interface":"front","average":{"1min":30.707000000000001,"5min":20.494,"15min":22.446999999999999},"min":{"1min":11.108000000000001,"5min":9.8379999999999992,"15min":9.6159999999999997},"max":{"1min":134.17400000000001,"5min":134.17400000000001,"15min":201.52799999999999},"last":30.442}]}]},{"osd":0,"up_from":16,"seq":68719477137,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58772,"kb_used_data":5580,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313068,"statfs":{"total":96636764160,"available":96576581632,"internally_reserved":0,"allocated":5713920,"data_stored":3095870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 14:26:15 2023","interfaces":[{"interface":"back","average":{"1min":19.928999999999998,"5min":17.440999999999999,"15min":16.654},"min":{"1min":6.7800000000000002,"5min":5.4980000000000002,"15min":4.9720000000000004},"max":{"1min":97.855999999999995,"5min":135.13800000000001,"15min":342.61399999999998},"last":11.712},{"interface":"front","average":{"1min":21.222000000000001,"5min":19.111999999999998,"15min":17.628},"min":{"1min":5.9409999999999998,"5min":5.8810000000000002,"15min":5.5960000000000001},"max":{"1min":144.221,"5min":144.221,"15min":337.48000000000002},"last":13.032}]},{"osd":2,"last update":"Fri Oct 6 14:25:21 2023","interfaces":[{"interface":"back","average":{"1min":34.204999999999998,"5min":19.474,"15min":19.213000000000001},"min":{"1min":8.8699999999999992,"5min":7.2809999999999997,"15min":5.9669999999999996},"max":{"1min":184.036,"5min":184.036,"15min":184.036},"last":11.177},{"interface":"front","average":{"1min":30.013999999999999,"5min":15.800000000000001,"15min":17.484999999999999},"min":{"1min":5.718,"5min":5.718,"15min":5.718},"max":{"1min":181.06700000000001,"5min":181.06700000000001,"15min":181.06700000000001},"last":6.6470000000000002}]},{"osd":3,"last update":"Fri Oct 6 14:25:42 2023","interfaces":[{"interface":"back","average":{"1min":30.332000000000001,"5min":24.734999999999999,"15min":25.606999999999999},"min":{"1min":7.5949999999999998,"5min":7.4139999999999997,"15min":7.4139999999999997},"max":{"1min":189.10499999999999,"5min":189.10499999999999,"15min":189.10499999999999},"last":13.619999999999999},{"interface":"front","average":{"1min":31.155000000000001,"5min":24.603999999999999,"15min":24.998000000000001},"min":{"1min":10.960000000000001,"5min":6.9850000000000003,"15min":6.9850000000000003},"max":{"1min":163.60900000000001,"5min":163.60900000000001,"15min":163.60900000000001},"last":16.693999999999999}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":3655.643,"5min":746.56799999999998,"15min":262.995},"min":{"1min":13.273999999999999,"5min":8.7420000000000009,"15min":6.9269999999999996},"max":{"1min":39913.453999999998,"5min":39913.453999999998,"15min":39913.453999999998},"last":15.192},{"interface":"front","average":{"1min":3644.9029999999998,"5min":742.30499999999995,"15min":261.23700000000002},"min":{"1min":7.2050000000000001,"5min":7.2050000000000001,"15min":7.2050000000000001},"max":{"1min":39912.421999999999,"5min":39912.421999999999,"15min":39912.421999999999},"last":9.0779999999999994}]},{"osd":5,"last update":"Fri Oct 6 14:25:49 2023","interfaces":[{"interface":"back","average":{"1min":15.067,"5min":16.125,"15min":71.585999999999999},"min":{"1min":8.2599999999999998,"5min":7.5209999999999999,"15min":7.5209999999999999},"max":{"1min":56.817,"5min":142.267,"15min":6380.991},"last":17.486000000000001},{"interface":"front","average":{"1min":24.161000000000001,"5min":20.405000000000001,"15min":66.221000000000004},"min":{"1min":10.645,"5min":9.9390000000000001,"15min":9.4550000000000001},"max":{"1min":133.797,"5min":145.59299999999999,"15min":6266.0159999999996},"last":12.458}]},{"osd":6,"last update":"Fri Oct 6 14:26:05 2023","interfaces":[{"interface":"back","average":{"1min":13.15,"5min":20.032,"15min":725.85500000000002},"min":{"1min":7.7789999999999999,"5min":7.1479999999999997,"15min":7.1479999999999997},"max":{"1min":26.087,"5min":177.73699999999999,"15min":46416.196000000004},"last":10.26},{"interface":"front","average":{"1min":15.784000000000001,"5min":22.670000000000002,"15min":724.05899999999997},"min":{"1min":10.669,"5min":8.6609999999999996,"15min":6.9459999999999997},"max":{"1min":21.206,"5min":181.625,"15min":46403.283000000003},"last":18.282}]},{"osd":7,"last update":"Fri Oct 6 14:25:31 2023","interfaces":[{"interface":"back","average":{"1min":38.594000000000001,"5min":23.141999999999999,"15min":1331.846},"min":{"1min":14.388,"5min":10.557,"15min":9.8209999999999997},"max":{"1min":178.86500000000001,"5min":178.86500000000001,"15min":67028.107999999993},"last":14.598000000000001},{"interface":"front","average":{"1min":31.856999999999999,"5min":20.099,"15min":1329.4469999999999},"min":{"1min":9.3140000000000001,"5min":8.4320000000000004,"15min":8.4320000000000004},"max":{"1min":178.38900000000001,"5min":178.38900000000001,"15min":67001.737999999998},"last":9.6379999999999999}]}]},{"osd":4,"up_from":430,"seq":1846835937654,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58704,"kb_used_data":5512,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313136,"statfs":{"total":96636764160,"available":96576651264,"internally_reserved":0,"allocated":5644288,"data_stored":3017805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":61.039999999999999,"5min":21.318999999999999,"15min":15.34},"min":{"1min":7.0979999999999999,"5min":4.508,"15min":4.085},"max":{"1min":469.096,"5min":469.096,"15min":469.096},"last":7.2140000000000004},{"interface":"front","average":{"1min":60.287999999999997,"5min":20.288,"15min":17.068999999999999},"min":{"1min":8.5890000000000004,"5min":4.758,"15min":4.5999999999999996},"max":{"1min":423.42000000000002,"5min":423.42000000000002,"15min":423.42000000000002},"last":7.75}]},{"osd":1,"last update":"Fri Oct 6 14:26:13 2023","interfaces":[{"interface":"back","average":{"1min":81.888000000000005,"5min":27.257000000000001,"15min":19.550000000000001},"min":{"1min":7.3289999999999997,"5min":5.9850000000000003,"15min":4.9390000000000001},"max":{"1min":469.72800000000001,"5min":469.72800000000001,"15min":469.72800000000001},"last":10.218999999999999},{"interface":"front","average":{"1min":77.108999999999995,"5min":26.469999999999999,"15min":21.765999999999998},"min":{"1min":5.9320000000000004,"5min":5.9320000000000004,"15min":5.9320000000000004},"max":{"1min":425.584,"5min":425.584,"15min":425.584},"last":10.827999999999999}]},{"osd":2,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":113.371,"5min":34.213000000000001,"15min":23.704999999999998},"min":{"1min":7.6429999999999998,"5min":6.8159999999999998,"15min":6.4770000000000003},"max":{"1min":530.30399999999997,"5min":530.30399999999997,"15min":530.30399999999997},"last":12.532999999999999},{"interface":"front","average":{"1min":117.30200000000001,"5min":37.220999999999997,"15min":26.550999999999998},"min":{"1min":10.667999999999999,"5min":7.0750000000000002,"15min":6.5780000000000003},"max":{"1min":530.86099999999999,"5min":530.86099999999999,"15min":530.86099999999999},"last":13.74}]},{"osd":3,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":58.695,"5min":25.254999999999999,"15min":19.239000000000001},"min":{"1min":11.391,"5min":9.9220000000000006,"15min":9.9220000000000006},"max":{"1min":470.80099999999999,"5min":470.80099999999999,"15min":470.80099999999999},"last":15.529999999999999},{"interface":"front","average":{"1min":49.429000000000002,"5min":22.123999999999999,"15min":17.120000000000001},"min":{"1min":6.7489999999999997,"5min":6.7489999999999997,"15min":6.7489999999999997},"max":{"1min":422.88,"5min":422.88,"15min":422.88},"last":9.0169999999999995}]},{"osd":5,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":58.246000000000002,"5min":23.637,"15min":76.748000000000005},"min":{"1min":8.8409999999999993,"5min":8.2230000000000008,"15min":5.9509999999999996},"max":{"1min":431.57900000000001,"5min":431.57900000000001,"15min":8813.2510000000002},"last":11.443},{"interface":"front","average":{"1min":62.423000000000002,"5min":22.936,"15min":77.950000000000003},"min":{"1min":6.5590000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"max":{"1min":537.68399999999997,"5min":537.68399999999997,"15min":8811.7579999999998},"last":13.138}]},{"osd":6,"last update":"Fri Oct 6 14:26:13 2023","interfaces":[{"interface":"back","average":{"1min":96.150999999999996,"5min":31.527000000000001,"15min":920.85000000000002},"min":{"1min":9.7840000000000007,"5min":6.9470000000000001,"15min":6.9470000000000001},"max":{"1min":563.85900000000004,"5min":563.85900000000004,"15min":44817.595000000001},"last":14.313000000000001},{"interface":"front","average":{"1min":84.043000000000006,"5min":30.527999999999999,"15min":922.12},"min":{"1min":12.715,"5min":7.5830000000000002,"15min":7.3789999999999996},"max":{"1min":471.80700000000002,"5min":471.80700000000002,"15min":44816.752999999997},"last":14.970000000000001}]},{"osd":7,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":50.648000000000003,"5min":23.149999999999999,"15min":1630.123},"min":{"1min":9.8119999999999994,"5min":9.7119999999999997,"15min":7.569},"max":{"1min":564.822,"5min":564.822,"15min":65382.432000000001},"last":18.446000000000002},{"interface":"front","average":{"1min":45.387999999999998,"5min":21.542999999999999,"15min":1627.549},"min":{"1min":13.016999999999999,"5min":7.8639999999999999,"15min":7.4059999999999997},"max":{"1min":433.30799999999999,"5min":433.30799999999999,"15min":65371.873},"last":20.347999999999999}]}]},{"osd":3,"up_from":834,"seq":3582002725217,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58716,"kb_used_data":5524,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313124,"statfs":{"total":96636764160,"available":96576638976,"internally_reserved":0,"allocated":5656576,"data_stored":3006177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":82,"apply_latency_ms":82,"commit_latency_ns":82000000,"apply_latency_ns":82000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":9.8170000000000002,"5min":26.867000000000001,"15min":32.284999999999997},"min":{"1min":5.6029999999999998,"5min":4.8559999999999999,"15min":4.8559999999999999},"max":{"1min":18.390999999999998,"5min":188.56700000000001,"15min":188.56700000000001},"last":22.681000000000001},{"interface":"front","average":{"1min":16.754999999999999,"5min":27.206,"15min":31.43},"min":{"1min":5.9870000000000001,"5min":5.827,"15min":5.827},"max":{"1min":135.715,"5min":214.209,"15min":214.209},"last":38.872999999999998}]},{"osd":1,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":63.271000000000001,"5min":37.026000000000003,"15min":34.704000000000001},"min":{"1min":7.7329999999999997,"5min":7.4409999999999998,"15min":7.4409999999999998},"max":{"1min":993.702,"5min":993.702,"15min":993.702},"last":122.386},{"interface":"front","average":{"1min":65.965999999999994,"5min":38.597999999999999,"15min":36.668999999999997},"min":{"1min":7.8520000000000003,"5min":6.6100000000000003,"15min":6.6100000000000003},"max":{"1min":991.50599999999997,"5min":991.50599999999997,"15min":991.50599999999997},"last":147.75800000000001}]},{"osd":2,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":21.853000000000002,"5min":29.605,"15min":34.101999999999997},"min":{"1min":8.9570000000000007,"5min":8.4199999999999999,"15min":8.4199999999999999},"max":{"1min":154.76599999999999,"5min":230.25399999999999,"15min":230.25399999999999},"last":52.75},{"interface":"front","average":{"1min":19.876000000000001,"5min":27.256,"15min":30.832000000000001},"min":{"1min":8.1959999999999997,"5min":8.1959999999999997,"15min":8.1959999999999997},"max":{"1min":95.603999999999999,"5min":165.553,"15min":165.553},"last":34.790999999999997}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":2736.415,"5min":573.53200000000004,"15min":216.17099999999999},"min":{"1min":10.223000000000001,"5min":7.1020000000000003,"15min":7.1020000000000003},"max":{"1min":40828.069000000003,"5min":40828.069000000003,"15min":40828.069000000003},"last":126.91200000000001},{"interface":"front","average":{"1min":18.699999999999999,"5min":28.707999999999998,"15min":32.601999999999997},"min":{"1min":8.6150000000000002,"5min":8.6150000000000002,"15min":8.6150000000000002},"max":{"1min":100.607,"5min":194.14599999999999,"15min":194.14599999999999},"last":83.572999999999993}]},{"osd":5,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":19.344000000000001,"5min":30.407,"15min":34.871000000000002},"min":{"1min":10.394,"5min":10.394,"15min":10.394},"max":{"1min":98.769000000000005,"5min":196.50200000000001,"15min":196.50200000000001},"last":36.985999999999997},{"interface":"front","average":{"1min":18.803000000000001,"5min":27.030999999999999,"15min":30.731000000000002},"min":{"1min":7.9329999999999998,"5min":7.9329999999999998,"15min":7.9329999999999998},"max":{"1min":111.729,"5min":177.739,"15min":177.739},"last":42.231999999999999}]},{"osd":6,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":20.492999999999999,"5min":38.835999999999999,"15min":47.399000000000001},"min":{"1min":9.6929999999999996,"5min":9.6929999999999996,"15min":9.6929999999999996},"max":{"1min":113.20399999999999,"5min":381.13,"15min":381.13},"last":127.839},{"interface":"front","average":{"1min":22.385999999999999,"5min":32.366999999999997,"15min":37.090000000000003},"min":{"1min":10.928000000000001,"5min":10.314,"15min":10.314},"max":{"1min":108.327,"5min":245.16200000000001,"15min":245.16200000000001},"last":109.922}]},{"osd":7,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":23.408999999999999,"5min":27.462,"15min":29.341999999999999},"min":{"1min":11.864000000000001,"5min":11.411,"15min":11.411},"max":{"1min":113.56699999999999,"5min":227.42599999999999,"15min":227.42599999999999},"last":81.742999999999995},{"interface":"front","average":{"1min":21.158000000000001,"5min":24.553999999999998,"15min":25.303999999999998},"min":{"1min":11.800000000000001,"5min":11.800000000000001,"15min":11.800000000000001},"max":{"1min":101.83499999999999,"5min":164.09899999999999,"15min":164.09899999999999},"last":52.259999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509831,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58660,"kb_used_data":5468,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313180,"statfs":{"total":96636764160,"available":96576696320,"internally_reserved":0,"allocated":5599232,"data_stored":2976840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":12.117000000000001,"5min":11.677,"15min":16.064},"min":{"1min":7.3819999999999997,"5min":5.452,"15min":5.3630000000000004},"max":{"1min":24.227,"5min":31.484000000000002,"15min":182.96600000000001},"last":37.688000000000002},{"interface":"front","average":{"1min":10.268000000000001,"5min":12.111000000000001,"15min":17.263000000000002},"min":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"max":{"1min":17.821000000000002,"5min":44.423000000000002,"15min":226.68600000000001},"last":27.803000000000001}]},{"osd":1,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":18.187000000000001,"5min":16.309999999999999,"15min":17.614000000000001},"min":{"1min":7.6710000000000003,"5min":7.3630000000000004,"15min":6.0970000000000004},"max":{"1min":100.065,"5min":101.874,"15min":148.20400000000001},"last":38.289000000000001},{"interface":"front","average":{"1min":12.387,"5min":14.256,"15min":17.227},"min":{"1min":6.6840000000000002,"5min":6.4630000000000001,"15min":6.3140000000000001},"max":{"1min":53.959000000000003,"5min":161.97499999999999,"15min":229.82499999999999},"last":51.273000000000003}]},{"osd":3,"last update":"Fri Oct 6 14:25:48 2023","interfaces":[{"interface":"back","average":{"1min":16.808,"5min":16.744,"15min":16.396000000000001},"min":{"1min":8.2769999999999992,"5min":8.2769999999999992,"15min":8.2769999999999992},"max":{"1min":31.460000000000001,"5min":33.399999999999999,"15min":33.399999999999999},"last":62.646999999999998},{"interface":"front","average":{"1min":13.861000000000001,"5min":14.6,"15min":14.781000000000001},"min":{"1min":8.6799999999999997,"5min":8.2509999999999994,"15min":8.2509999999999994},"max":{"1min":28.713000000000001,"5min":29.898,"15min":29.898},"last":64.054000000000002}]},{"osd":4,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":14883.07,"5min":2990.154,"15min":1010.401},"min":{"1min":6.8150000000000004,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":38405.120000000003,"5min":38405.120000000003,"15min":38405.120000000003},"last":70.638000000000005},{"interface":"front","average":{"1min":14785.589,"5min":2970.7600000000002,"15min":1006.212},"min":{"1min":7.8520000000000003,"5min":7.6550000000000002,"15min":7.6550000000000002},"max":{"1min":38264.116000000002,"5min":38264.116000000002,"15min":38264.116000000002},"last":28.445}]},{"osd":5,"last update":"Fri Oct 6 14:25:51 2023","interfaces":[{"interface":"back","average":{"1min":16.574999999999999,"5min":16.280999999999999,"15min":92.066999999999993},"min":{"1min":8.2560000000000002,"5min":8.2560000000000002,"15min":7.6589999999999998},"max":{"1min":32.328000000000003,"5min":50.210000000000001,"15min":9721.0490000000009},"last":73.159000000000006},{"interface":"front","average":{"1min":14.137,"5min":16.710999999999999,"15min":96.823999999999998},"min":{"1min":8.6099999999999994,"5min":8.6099999999999994,"15min":8.6099999999999994},"max":{"1min":24.001000000000001,"5min":73.320999999999998,"15min":9793.7890000000007},"last":77.673000000000002}]},{"osd":6,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":14.999000000000001,"5min":17.789000000000001,"15min":1309.6320000000001},"min":{"1min":8.8689999999999998,"5min":8.8689999999999998,"15min":8.8689999999999998},"max":{"1min":23.431999999999999,"5min":44.606999999999999,"15min":44531.741000000002},"last":34.725999999999999},{"interface":"front","average":{"1min":16.434000000000001,"5min":16.972000000000001,"15min":1305.6900000000001},"min":{"1min":7.5330000000000004,"5min":7.141,"15min":7.141},"max":{"1min":49.576999999999998,"5min":49.576999999999998,"15min":44501.423000000003},"last":76.936000000000007}]},{"osd":7,"last update":"Fri Oct 6 14:25:34 2023","interfaces":[{"interface":"back","average":{"1min":20.236000000000001,"5min":20.585000000000001,"15min":1733.5730000000001},"min":{"1min":12.177,"5min":10.631,"15min":8.6189999999999998},"max":{"1min":38.399999999999999,"5min":61.658999999999999,"15min":69773.953999999998},"last":73.903000000000006},{"interface":"front","average":{"1min":21.065999999999999,"5min":18.300999999999998,"15min":1723.3030000000001},"min":{"1min":11.403,"5min":9.5909999999999993,"15min":8.5429999999999993},"max":{"1min":31.567,"5min":34.923000000000002,"15min":69547.547999999995},"last":78.433999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62840,"kb_used_data":5552,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94309000,"statfs":{"total":96636764160,"available":96572416000,"internally_reserved":0,"allocated":5685248,"data_stored":3054367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":56,"apply_latency_ms":56,"commit_latency_ns":56000000,"apply_latency_ns":56000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":8.6549999999999994,"5min":11.162000000000001,"15min":18.341000000000001},"min":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.4050000000000002},"max":{"1min":12.99,"5min":94.040999999999997,"15min":574.78800000000001},"last":5.0999999999999996},{"interface":"front","average":{"1min":11.332000000000001,"5min":15.573,"15min":21.341999999999999},"min":{"1min":7.5309999999999997,"5min":7.5309999999999997,"15min":4.5140000000000002},"max":{"1min":17.783999999999999,"5min":130.036,"15min":522.72699999999998},"last":7.6779999999999999}]},{"osd":1,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.231,"5min":27.030999999999999,"15min":24.16},"min":{"1min":7.3150000000000004,"5min":5.4059999999999997,"15min":5.4059999999999997},"max":{"1min":14.238,"5min":1271.6179999999999,"15min":1271.6179999999999},"last":9.1989999999999998},{"interface":"front","average":{"1min":12.869,"5min":30.803000000000001,"15min":27.870999999999999},"min":{"1min":9.9900000000000002,"5min":6.8639999999999999,"15min":6.8639999999999999},"max":{"1min":17.446999999999999,"5min":1322.71,"15min":1322.71},"last":12.44}]},{"osd":2,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.842000000000001,"5min":14.388999999999999,"15min":19.920999999999999},"min":{"1min":9.4809999999999999,"5min":6.1660000000000004,"15min":5.532},"max":{"1min":30.937999999999999,"5min":101.197,"15min":525.81600000000003},"last":16.143999999999998},{"interface":"front","average":{"1min":11.125999999999999,"5min":17.469999999999999,"15min":24.055},"min":{"1min":6.7409999999999997,"5min":6.4509999999999996,"15min":6.4509999999999996},"max":{"1min":27.02,"5min":99.492999999999995,"15min":468.70400000000001},"last":9.9740000000000002}]},{"osd":3,"last update":"Fri Oct 6 14:25:41 2023","interfaces":[{"interface":"back","average":{"1min":15.800000000000001,"5min":20.021000000000001,"15min":20.113},"min":{"1min":6.8470000000000004,"5min":6.8470000000000004,"15min":6.8470000000000004},"max":{"1min":40.512,"5min":126.55500000000001,"15min":126.55500000000001},"last":17.16},{"interface":"front","average":{"1min":14.677,"5min":21.684999999999999,"15min":23.826000000000001},"min":{"1min":6.0389999999999997,"5min":6.0389999999999997,"15min":6.0389999999999997},"max":{"1min":49.473999999999997,"5min":236.27000000000001,"15min":236.27000000000001},"last":11.718}]},{"osd":4,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15821.066000000001,"5min":3179.7730000000001,"15min":1076.7719999999999},"min":{"1min":10.141,"5min":9.1189999999999998,"15min":8.7490000000000006},"max":{"1min":38720.860999999997,"5min":38720.860999999997,"15min":38720.860999999997},"last":14.757},{"interface":"front","average":{"1min":15816.687,"5min":3177.5610000000001,"15min":1074.1969999999999},"min":{"1min":6.782,"5min":6.782,"15min":6.782},"max":{"1min":38719.173000000003,"5min":38719.173000000003,"15min":38719.173000000003},"last":13.971}]},{"osd":5,"last update":"Fri Oct 6 14:25:51 2023","interfaces":[{"interface":"back","average":{"1min":17.318999999999999,"5min":21.241,"15min":107.18600000000001},"min":{"1min":8.8800000000000008,"5min":8.8800000000000008,"15min":8.8800000000000008},"max":{"1min":43.317999999999998,"5min":219.06200000000001,"15min":8032.4189999999999},"last":11.210000000000001},{"interface":"front","average":{"1min":18.963999999999999,"5min":21.207999999999998,"15min":109.759},"min":{"1min":7.4409999999999998,"5min":7.1459999999999999,"15min":7.1459999999999999},"max":{"1min":69.081000000000003,"5min":171.041,"15min":8030.5749999999998},"last":16.884}]},{"osd":7,"last update":"Fri Oct 6 14:25:37 2023","interfaces":[{"interface":"back","average":{"1min":20.959,"5min":19.791,"15min":1130.614},"min":{"1min":11.01,"5min":11.01,"15min":11.01},"max":{"1min":45.648000000000003,"5min":93.355999999999995,"15min":69750.712},"last":18.297000000000001},{"interface":"front","average":{"1min":16.279,"5min":14.369999999999999,"15min":1124.646},"min":{"1min":7.1769999999999996,"5min":6.1070000000000002,"15min":5.984},"max":{"1min":40.57,"5min":121.146,"15min":69749.794999999998},"last":10.629}]}]},{"osd":5,"up_from":659,"seq":2830383448439,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77156,"kb_used_data":5916,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94294684,"statfs":{"total":96636764160,"available":96557756416,"internally_reserved":0,"allocated":6057984,"data_stored":3418788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":65,"apply_latency_ms":65,"commit_latency_ns":65000000,"apply_latency_ns":65000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":10.645,"5min":13.339,"15min":38.990000000000002},"min":{"1min":4.6189999999999998,"5min":4.3339999999999996,"15min":4.1879999999999997},"max":{"1min":22.547999999999998,"5min":173.54599999999999,"15min":680.28200000000004},"last":7.141},{"interface":"front","average":{"1min":7.4779999999999998,"5min":11.548999999999999,"15min":29.652000000000001},"min":{"1min":4.7430000000000003,"5min":4.5579999999999998,"15min":4.5579999999999998},"max":{"1min":16.622,"5min":154.36199999999999,"15min":608.46199999999999},"last":14.422000000000001}]},{"osd":1,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":45.543999999999997,"5min":20.469000000000001,"15min":43.674999999999997},"min":{"1min":6.3620000000000001,"5min":5.4930000000000003,"15min":5.4930000000000003},"max":{"1min":633.78700000000003,"5min":633.78700000000003,"15min":633.78700000000003},"last":12.051},{"interface":"front","average":{"1min":46.32,"5min":23.73,"15min":46.381},"min":{"1min":10.032,"5min":5.9000000000000004,"15min":5.8259999999999996},"max":{"1min":598.60000000000002,"5min":598.60000000000002,"15min":701.19399999999996},"last":11.423}]},{"osd":2,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":14.177,"5min":14.871,"15min":44.731999999999999},"min":{"1min":8.218,"5min":6.8369999999999997,"15min":6.6029999999999998},"max":{"1min":27.632000000000001,"5min":87.468000000000004,"15min":603.024},"last":16.187999999999999},{"interface":"front","average":{"1min":15.343,"5min":17.352,"15min":49.799999999999997},"min":{"1min":10.210000000000001,"5min":8.1289999999999996,"15min":8.1289999999999996},"max":{"1min":29.491,"5min":84.820999999999998,"15min":690.71799999999996},"last":19.492000000000001}]},{"osd":3,"last update":"Fri Oct 6 14:25:42 2023","interfaces":[{"interface":"back","average":{"1min":15.289999999999999,"5min":28.317,"15min":32.777999999999999},"min":{"1min":9.0920000000000005,"5min":8.1690000000000005,"15min":8.1690000000000005},"max":{"1min":31.006,"5min":264.37900000000002,"15min":264.37900000000002},"last":20.670000000000002},{"interface":"front","average":{"1min":16.370000000000001,"5min":27.132999999999999,"15min":30.166},"min":{"1min":11.48,"5min":9.0990000000000002,"15min":9.0990000000000002},"max":{"1min":33.726999999999997,"5min":173.858,"15min":173.858},"last":18.087}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":3491.913,"5min":712.81600000000003,"15min":274.36799999999999},"min":{"1min":10.141,"5min":7.9340000000000002,"15min":7.9340000000000002},"max":{"1min":38256.345000000001,"5min":38256.345000000001,"15min":38256.345000000001},"last":22.091999999999999},{"interface":"front","average":{"1min":3489.7570000000001,"5min":714.00800000000004,"15min":281.79899999999998},"min":{"1min":13.077999999999999,"5min":9.1750000000000007,"15min":8.1839999999999993},"max":{"1min":38222.745999999999,"5min":38222.745999999999,"15min":38222.745999999999},"last":12.853}]},{"osd":6,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":15.797000000000001,"5min":18.550999999999998,"15min":47.345999999999997},"min":{"1min":10.766,"5min":7.7300000000000004,"15min":7.7300000000000004},"max":{"1min":27.385000000000002,"5min":108.053,"15min":607.87400000000002},"last":16.995999999999999},{"interface":"front","average":{"1min":17.838000000000001,"5min":21.890000000000001,"15min":51.551000000000002},"min":{"1min":14.509,"5min":10.855,"15min":7.1040000000000001},"max":{"1min":26.152000000000001,"5min":91.733000000000004,"15min":739.33100000000002},"last":20.09}]},{"osd":7,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":17.960999999999999,"5min":21.77,"15min":47.536000000000001},"min":{"1min":10.858000000000001,"5min":9.3569999999999993,"15min":7.0350000000000001},"max":{"1min":32.581000000000003,"5min":90.037000000000006,"15min":609.59799999999996},"last":15.159000000000001},{"interface":"front","average":{"1min":9.0069999999999997,"5min":13.292,"15min":44.540999999999997},"min":{"1min":6.4420000000000002,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":13.343999999999999,"5min":77.034999999999997,"15min":610.43799999999999},"last":10.16}]}]},{"osd":7,"up_from":13,"seq":55834575232,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62840,"kb_used_data":5552,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94309000,"statfs":{"total":96636764160,"available":96572416000,"internally_reserved":0,"allocated":5685248,"data_stored":3067198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:25:26 2023","interfaces":[{"interface":"back","average":{"1min":8.4930000000000003,"5min":10.589,"15min":33.819000000000003},"min":{"1min":4.907,"5min":4.742,"15min":4.2539999999999996},"max":{"1min":14.169,"5min":46.529000000000003,"15min":930.62099999999998},"last":8.4350000000000005},{"interface":"front","average":{"1min":14.048,"5min":11.707000000000001,"15min":31.795999999999999},"min":{"1min":6.6950000000000003,"5min":4.8079999999999998,"15min":4.8079999999999998},"max":{"1min":69.477999999999994,"5min":69.477999999999994,"15min":879.35199999999998},"last":8.0879999999999992}]},{"osd":1,"last update":"Fri Oct 6 14:25:26 2023","interfaces":[{"interface":"back","average":{"1min":70.927000000000007,"5min":26.693999999999999,"15min":40.520000000000003},"min":{"1min":5.4790000000000001,"5min":5.4790000000000001,"15min":5.4790000000000001},"max":{"1min":1094.5889999999999,"5min":1094.5889999999999,"15min":1094.5889999999999},"last":14.412000000000001},{"interface":"front","average":{"1min":69.433000000000007,"5min":25.927,"15min":25.18},"min":{"1min":8.3170000000000002,"5min":6.2270000000000003,"15min":5.4539999999999997},"max":{"1min":1057.856,"5min":1057.856,"15min":1057.856},"last":17.975000000000001}]},{"osd":2,"last update":"Fri Oct 6 14:25:26 2023","interfaces":[{"interface":"back","average":{"1min":16.998000000000001,"5min":15.606999999999999,"15min":24.332000000000001},"min":{"1min":9.0139999999999993,"5min":6.5149999999999997,"15min":6.202},"max":{"1min":75.614000000000004,"5min":75.823999999999998,"15min":316.84399999999999},"last":9.9169999999999998},{"interface":"front","average":{"1min":18.135999999999999,"5min":16.129999999999999,"15min":40.203000000000003},"min":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"max":{"1min":78.471999999999994,"5min":98.183000000000007,"15min":931.94500000000005},"last":16.071999999999999}]},{"osd":3,"last update":"Fri Oct 6 14:25:47 2023","interfaces":[{"interface":"back","average":{"1min":14.034000000000001,"5min":20.844000000000001,"15min":22.948},"min":{"1min":6.9969999999999999,"5min":6.9969999999999999,"15min":6.9969999999999999},"max":{"1min":40.936,"5min":184.93700000000001,"15min":184.93700000000001},"last":13.608000000000001},{"interface":"front","average":{"1min":13.778,"5min":18.068999999999999,"15min":18.817},"min":{"1min":8.2010000000000005,"5min":7.6699999999999999,"15min":7.6699999999999999},"max":{"1min":32.956000000000003,"5min":77.144000000000005,"15min":77.144000000000005},"last":16.684999999999999}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":2132.1480000000001,"5min":439.87099999999998,"15min":167.66999999999999},"min":{"1min":7.8869999999999996,"5min":5.9429999999999996,"15min":5.9429999999999996},"max":{"1min":40210.642,"5min":40210.642,"15min":40210.642},"last":8.7219999999999995},{"interface":"front","average":{"1min":2144.9340000000002,"5min":442.64400000000001,"15min":163.51599999999999},"min":{"1min":11.497999999999999,"5min":7.4180000000000001,"15min":6.9279999999999999},"max":{"1min":40377.220000000001,"5min":40377.220000000001,"15min":40377.220000000001},"last":15.115}]},{"osd":5,"last update":"Fri Oct 6 14:25:50 2023","interfaces":[{"interface":"back","average":{"1min":17.524000000000001,"5min":18.869,"15min":51.777000000000001},"min":{"1min":8.4939999999999998,"5min":8.1229999999999993,"15min":7.6909999999999998},"max":{"1min":70.796000000000006,"5min":149.61799999999999,"15min":6750.3770000000004},"last":12.308},{"interface":"front","average":{"1min":19.053999999999998,"5min":19.542000000000002,"15min":52.655000000000001},"min":{"1min":7.9029999999999996,"5min":6.9290000000000003,"15min":6.9290000000000003},"max":{"1min":83.019000000000005,"5min":174.839,"15min":6875.5780000000004},"last":12.619999999999999}]},{"osd":6,"last update":"Fri Oct 6 14:25:26 2023","interfaces":[{"interface":"back","average":{"1min":20.823,"5min":17.948,"15min":27.071999999999999},"min":{"1min":11.436999999999999,"5min":7.9509999999999996,"15min":7.7290000000000001},"max":{"1min":75.013999999999996,"5min":75.013999999999996,"15min":380.74000000000001},"last":12.039999999999999},{"interface":"front","average":{"1min":23.314,"5min":19.940999999999999,"15min":44.384999999999998},"min":{"1min":9.1120000000000001,"5min":8.9489999999999998,"15min":8.9489999999999998},"max":{"1min":83.512,"5min":120.83799999999999,"15min":935.49300000000005},"last":17.314}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T14:26:25.713 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T14:26:25.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.817+0000 7f07eb4b6640 10 monclient: get_monmap_and_config 2023-10-06T14:26:25.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.817+0000 7f07eb4b6640 10 monclient: build_initial_monmap 2023-10-06T14:26:25.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.817+0000 7f07eb4b6640 10 monclient: monmap: 2023-10-06T14:26:25.819 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:25.819 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:25.819 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:25.818197+0000 2023-10-06T14:26:25.819 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:25.818197+0000 2023-10-06T14:26:25.819 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:25.820 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:25.820 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:25.820 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:25.820 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:25.820 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:25.820 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:25.820 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:25.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.817+0000 7f07eb4b6640 1 Processor -- start 2023-10-06T14:26:25.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 1 -- start start 2023-10-06T14:26:25.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 10 monclient: init 2023-10-06T14:26:25.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:25.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 10 monclient: _add_conns ranks=[1,2,3,4,0] 2023-10-06T14:26:25.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f07e40b3990 addr v1:172.21.15.83:6789/0 2023-10-06T14:26:25.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f07e40b3d60 0x7f07e40af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:25.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 10 monclient(hunting): picked mon.noname-b con 0x7f07e40b3d60 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:25.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f07e40b00c0 0x7f07e40b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:25.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 10 monclient(hunting): picked mon.noname-a con 0x7f07e40b00c0 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e40aebe0 con 0x7f07e40b3990 2023-10-06T14:26:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e40afd50 con 0x7f07e40b00c0 2023-10-06T14:26:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e4140dd0 con 0x7f07e40b3d60 2023-10-06T14:26:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.818+0000 7f07eb4b6640 10 monclient(hunting): authenticate will time out at 3314.005038s 2023-10-06T14:26:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.819+0000 7f07e8a51640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f07e40b3d60 0x7f07e40af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.819+0000 7f07e9252640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f07e40b3990 0x7f07e4141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:42936/0 (socket says 172.21.15.19:42936) 2023-10-06T14:26:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.819+0000 7f07e9252640 1 -- 172.21.15.19:0/922501076 learned_addr learned my addr 172.21.15.19:0/922501076 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T14:26:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.819+0000 7f07e8a51640 10 monclient(hunting): get_auth_request con 0x7f07e40b3d60 auth_method 0 2023-10-06T14:26:25.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.819+0000 7f07e8a51640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:25.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.819+0000 7f07e8a51640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:25.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.819+0000 7f07e8a51640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:25.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.820+0000 7f07e8a51640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:25.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.820+0000 7f07e8a51640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:25.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.820+0000 7f07e8a51640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:25.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.820+0000 7f07e9a53640 1 --2- 172.21.15.19:0/922501076 >> v2:172.21.15.19:3300/0 conn(0x7f07e40b00c0 0x7f07e40b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:25.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e9a53640 10 monclient(hunting): get_auth_request con 0x7f07e40b00c0 auth_method 0 2023-10-06T14:26:25.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e9a53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:25.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e9a53640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:25.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e9a53640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:25.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e9a53640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:25.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e9a53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:25.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e9a53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:25.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07d7fff640 1 -- 172.21.15.19:0/922501076 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195362432 0 0) 0x7f07e40aebe0 con 0x7f07e40b3990 2023-10-06T14:26:25.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07d7fff640 10 monclient(hunting): _negotiate 2023-10-06T14:26:25.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07d7fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:25.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07d7fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:25.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07d7fff640 10 monclient(hunting): my global_id is 20725 2023-10-06T14:26:25.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07d7fff640 1 -- 172.21.15.19:0/922501076 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07d0003520 con 0x7f07e40b3990 2023-10-06T14:26:25.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e8a51640 10 monclient(hunting): handle_auth_done global_id 17114 payload 274 2023-10-06T14:26:25.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e8a51640 1 -- 172.21.15.19:0/922501076 >> v2:172.21.15.19:3300/0 conn(0x7f07e40b00c0 msgr2=0x7f07e40b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:25.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e8a51640 1 --2- 172.21.15.19:0/922501076 >> v2:172.21.15.19:3300/0 conn(0x7f07e40b00c0 0x7f07e40b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e8a51640 1 -- 172.21.15.19:0/922501076 >> v1:172.21.15.83:6789/0 conn(0x7f07e40b3990 legacy=0x7f07e4141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:25.830 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e8a51640 10 monclient: _finish_hunting 0 2023-10-06T14:26:25.830 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e8a51640 1 monclient: found mon.noname-b 2023-10-06T14:26:25.830 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e8a51640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:25.830 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e8a51640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T14:26:25.830 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.821+0000 7f07e8a51640 1 -- 172.21.15.19:0/922501076 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e4141070 con 0x7f07e40b3d60 2023-10-06T14:26:25.830 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.823+0000 7f07e8a51640 1 --2- 172.21.15.19:0/922501076 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f07e40b3d60 0x7f07e40af810 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 crypto rx=0x7f07e00097a0 tx=0x7f07e002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e21689f2631e6fff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:25.834 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07d7fff640 1 -- 172.21.15.19:0/922501076 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e002e070 con 0x7f07e40b3d60 2023-10-06T14:26:25.834 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:25.834 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07d7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:25.834 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07d7fff640 10 monclient: dump: 2023-10-06T14:26:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:25.835 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07d7fff640 10 monclient: _finish_auth 0 2023-10-06T14:26:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07d7fff640 10 monclient: _check_auth_tickets 2023-10-06T14:26:25.836 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:25.836 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07d7fff640 1 -- 172.21.15.19:0/922501076 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e00315a0 con 0x7f07e40b3d60 2023-10-06T14:26:25.836 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07eb4b6640 5 monclient: authenticate success, global_id 17114 2023-10-06T14:26:25.836 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07eb4b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:25.836 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07d7fff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:25.837 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07d7fff640 1 -- 172.21.15.19:0/922501076 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0031930 con 0x7f07e40b3d60 2023-10-06T14:26:25.837 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07eb4b6640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:25.837 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07eb4b6640 4 set_mon_vals no callback set 2023-10-06T14:26:25.837 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:25.837 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07d7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:25.837 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07d7fff640 10 monclient: dump: 2023-10-06T14:26:25.838 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:25.838 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:25.838 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.839 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.839 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:25.839 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:25.839 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:25.839 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:25.839 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:25.839 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:25.839 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07eb4b6640 10 monclient: shutdown 2023-10-06T14:26:25.839 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07eb4b6640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:25.839 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07eb4b6640 1 -- 172.21.15.19:0/922501076 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f07e40b3d60 msgr2=0x7f07e40af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:25.839 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.827+0000 7f07eb4b6640 1 --2- 172.21.15.19:0/922501076 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f07e40b3d60 0x7f07e40af810 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 crypto rx=0x7f07e00097a0 tx=0x7f07e002b840 comp rx=0 tx=0).stop 2023-10-06T14:26:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.828+0000 7f07eb4b6640 1 -- 172.21.15.19:0/922501076 shutdown_connections 2023-10-06T14:26:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.828+0000 7f07eb4b6640 1 --2- 172.21.15.19:0/922501076 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f07e40b3d60 0x7f07e40af810 unknown :-1 s=CLOSED pgs=3116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.828+0000 7f07eb4b6640 1 --2- 172.21.15.19:0/922501076 >> v2:172.21.15.19:3300/0 conn(0x7f07e40b00c0 0x7f07e40b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.828+0000 7f07eb4b6640 1 -- 172.21.15.19:0/922501076 >> 172.21.15.19:0/922501076 conn(0x7f07e40b6e00 msgr2=0x7f07e40b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.828+0000 7f07eb4b6640 1 -- 172.21.15.19:0/922501076 shutdown_connections 2023-10-06T14:26:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.828+0000 7f07eb4b6640 1 -- 172.21.15.19:0/922501076 wait complete. 2023-10-06T14:26:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.828+0000 7f07eb4b6640 10 monclient: build_initial_monmap 2023-10-06T14:26:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.828+0000 7f07eb4b6640 10 monclient: monmap: 2023-10-06T14:26:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:25.842 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 1 librados: starting msgr at 2023-10-06T14:26:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 1 librados: starting objecter 2023-10-06T14:26:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 1 Processor -- start 2023-10-06T14:26:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 1 -- start start 2023-10-06T14:26:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 1 librados: setting wanted keys 2023-10-06T14:26:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 1 librados: calling monclient init 2023-10-06T14:26:25.844 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 10 monclient: init 2023-10-06T14:26:25.844 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:25.844 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T14:26:25.844 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40b00c0 0x7f07e413d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:25.844 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 10 monclient(hunting): picked mon.noname-a con 0x7f07e40b00c0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:25.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07e40b3990 0x7f07e40a9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:25.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 10 monclient(hunting): picked mon.noname-b con 0x7f07e40b3990 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:25.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f07e40b3d60 0x7f07e40a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:25.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 10 monclient(hunting): picked mon.noname-c con 0x7f07e40b3d60 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:25.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:25.845 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40a9eb0 con 0x7f07e40b00c0 2023-10-06T14:26:25.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:25.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40abcf0 con 0x7f07e40b3990 2023-10-06T14:26:25.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:25.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e413de40 con 0x7f07e40b3d60 2023-10-06T14:26:25.846 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.829+0000 7f07eb4b6640 10 monclient(hunting): authenticate will time out at 3314.016239s 2023-10-06T14:26:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e9a53640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f07e40b3d60 0x7f07e40a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e9252640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40b00c0 0x7f07e413d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e8a51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07e40b3990 0x7f07e40a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e9a53640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f07e40b3d60 0x7f07e40a8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40206/0 (socket says 172.21.15.19:40206) 2023-10-06T14:26:25.848 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e9a53640 1 -- 172.21.15.19:0/2529242516 learned_addr learned my addr 172.21.15.19:0/2529242516 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:25.848 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e9a53640 10 monclient(hunting): get_auth_request con 0x7f07e40b3d60 auth_method 0 2023-10-06T14:26:25.848 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e9a53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:25.848 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e9a53640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:25.848 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e9a53640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:25.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e9252640 10 monclient(hunting): get_auth_request con 0x7f07e40b00c0 auth_method 0 2023-10-06T14:26:25.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e9252640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:25.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e9252640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:25.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e9252640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:25.849 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e8a51640 10 monclient(hunting): get_auth_request con 0x7f07e40b3990 auth_method 0 2023-10-06T14:26:25.850 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e8a51640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:25.850 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e8a51640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:25.850 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.830+0000 7f07e8a51640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:25.850 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.831+0000 7f07e9a53640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:25.850 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.831+0000 7f07e9a53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:25.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.831+0000 7f07e9a53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:25.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.831+0000 7f07e8a51640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:25.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.831+0000 7f07e8a51640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:25.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.831+0000 7f07e8a51640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:25.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.831+0000 7f07e9252640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:25.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.831+0000 7f07e9252640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:25.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.831+0000 7f07e9252640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:25.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.833+0000 7f07e8a51640 10 monclient(hunting): handle_auth_done global_id 20728 payload 824 2023-10-06T14:26:25.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.833+0000 7f07e8a51640 1 -- 172.21.15.19:0/2529242516 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f07e40b3d60 msgr2=0x7f07e40a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:25.856 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.833+0000 7f07e8a51640 1 --2- 172.21.15.19:0/2529242516 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f07e40b3d60 0x7f07e40a8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.856 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.833+0000 7f07e8a51640 1 -- 172.21.15.19:0/2529242516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40b00c0 msgr2=0x7f07e413d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:25.856 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.833+0000 7f07e8a51640 1 --2- 172.21.15.19:0/2529242516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40b00c0 0x7f07e413d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:25.856 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.833+0000 7f07e8a51640 10 monclient: _finish_hunting 0 2023-10-06T14:26:25.856 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.833+0000 7f07e8a51640 1 monclient: found mon.noname-b 2023-10-06T14:26:25.856 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.833+0000 7f07e8a51640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:25.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.833+0000 7f07e8a51640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T14:26:25.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.833+0000 7f07e8a51640 1 -- 172.21.15.19:0/2529242516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e0009400 con 0x7f07e40b3990 2023-10-06T14:26:25.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.833+0000 7f07e9a53640 1 --2- 172.21.15.19:0/2529242516 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f07e40b3d60 0x7f07e40a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T14:26:25.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.836+0000 7f07e8a51640 1 --2- 172.21.15.19:0/2529242516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07e40b3990 0x7f07e40a9970 secure :-1 s=READY pgs=5105 cs=0 l=1 rev1=1 crypto rx=0x7f07e4140f10 tx=0x7f07e00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:25.857 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.836+0000 7f07d5ffb640 10 client.?.objecter ms_handle_connect 0x7f07e40b3990 2023-10-06T14:26:25.858 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.836+0000 7f07d5ffb640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:25.858 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07d5ffb640 1 -- 172.21.15.19:0/2529242516 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0032840 con 0x7f07e40b3990 2023-10-06T14:26:25.858 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:25.858 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07d5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:25.858 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07d5ffb640 10 monclient: dump: 2023-10-06T14:26:25.859 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:25.859 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:25.859 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.859 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.859 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:25.860 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:25.860 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:25.860 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:25.860 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:25.860 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:25.861 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07d5ffb640 10 monclient: _finish_auth 0 2023-10-06T14:26:25.861 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07d5ffb640 10 monclient: _check_auth_tickets 2023-10-06T14:26:25.861 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:25.861 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07eb4b6640 5 monclient: authenticate success, global_id 20728 2023-10-06T14:26:25.861 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07eb4b6640 10 monclient: _renew_subs 2023-10-06T14:26:25.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07eb4b6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:25.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07eb4b6640 1 -- 172.21.15.19:0/2529242516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e40a84f0 con 0x7f07e40b3990 2023-10-06T14:26:25.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07eb4b6640 10 client.20728.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:25.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07eb4b6640 10 monclient: _renew_subs 2023-10-06T14:26:25.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07eb4b6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:25.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07eb4b6640 1 -- 172.21.15.19:0/2529242516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e40ac6f0 con 0x7f07e40b3990 2023-10-06T14:26:25.863 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.838+0000 7f07eb4b6640 1 librados: init done 2023-10-06T14:26:25.863 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.839+0000 7f07eb4b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:25.863 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.839+0000 7f07eb4b6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:25.863 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.839+0000 7f07eb4b6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:25.863 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.839+0000 7f07eb4b6640 1 -- 172.21.15.19:0/2529242516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e40a8730 con 0x7f07e40b3990 2023-10-06T14:26:25.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.839+0000 7f07d5ffb640 1 -- 172.21.15.19:0/2529242516 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e00329a0 con 0x7f07e40b3990 2023-10-06T14:26:25.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.839+0000 7f07d5ffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:25.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.839+0000 7f07d5ffb640 1 -- 172.21.15.19:0/2529242516 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0032d10 con 0x7f07e40b3990 2023-10-06T14:26:25.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.839+0000 7f07d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:25.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.839+0000 7f07d5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:25.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.839+0000 7f07d5ffb640 10 monclient: dump: 2023-10-06T14:26:25.865 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:25.865 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:25.865 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.865 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:25.866 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:25.866 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:25.866 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:25.866 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:25.866 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:25.867 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:25.867 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:25.839+0000 7f07d77fe640 4 set_mon_vals no callback set 2023-10-06T14:26:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.079+0000 7f07d5ffb640 1 -- 172.21.15.19:0/2529242516 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f07e004d3e0 con 0x7f07e40b3990 2023-10-06T14:26:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.079+0000 7f07d5ffb640 1 --2- 172.21.15.19:0/2529242516 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f07c806c1f0 0x7f07c806e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.079+0000 7f07e9252640 1 --2- 172.21.15.19:0/2529242516 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f07c806c1f0 0x7f07c806e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.079+0000 7f07e9252640 10 monclient: get_auth_request con 0x7f07c806c1f0 auth_method 0 2023-10-06T14:26:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.080+0000 7f07e9252640 1 --2- 172.21.15.19:0/2529242516 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f07c806c1f0 0x7f07c806e6a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f07d80011e0 tx=0x7f07d8007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.080+0000 7f07d5ffb640 10 client.20728.objecter ms_handle_connect 0x7f07c806c1f0 2023-10-06T14:26:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.082+0000 7f07d5ffb640 1 -- 172.21.15.19:0/2529242516 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(969..969 src has 595..969) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f07e0079f00 con 0x7f07e40b3990 2023-10-06T14:26:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.082+0000 7f07d5ffb640 10 client.20728.objecter ms_dispatch 0x7f07e4000b90 osd_map(969..969 src has 595..969) v4 2023-10-06T14:26:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.082+0000 7f07d5ffb640 3 client.20728.objecter handle_osd_map got epochs [969,969] > 0 2023-10-06T14:26:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.082+0000 7f07d5ffb640 3 client.20728.objecter handle_osd_map decoding full epoch 969 2023-10-06T14:26:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.082+0000 7f07d5ffb640 20 client.20728.objecter dump_active .. 0 homeless 2023-10-06T14:26:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.092+0000 7f07d5ffb640 1 -- 172.21.15.19:0/2529242516 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f07e0076240 con 0x7f07e40b3990 2023-10-06T14:26:26.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.092+0000 7f07d5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:26.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.092+0000 7f07d5ffb640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:26:26.270 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.268+0000 7f07eb4b6640 1 -- 172.21.15.19:0/2529242516 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f07e40af110 con 0x7f07c806c1f0 2023-10-06T14:26:26.275 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.273+0000 7f07d5ffb640 1 -- 172.21.15.19:0/2529242516 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+151924 (secure 0 0 0) 0x7f07d80014d0 con 0x7f07c806c1f0 2023-10-06T14:26:26.277 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T14:26:26.278 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2023-10-06T14:26:26.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 10 librados: watch_flush enter 2023-10-06T14:26:26.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 10 librados: watch_flush exit 2023-10-06T14:26:26.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 20 client.20728.objecter shutdown clearing up homeless session... 2023-10-06T14:26:26.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 10 client.20728.objecter successfully canceled tick 2023-10-06T14:26:26.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 1 -- 172.21.15.19:0/2529242516 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f07c806c1f0 msgr2=0x7f07c806e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:26.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 1 --2- 172.21.15.19:0/2529242516 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f07c806c1f0 0x7f07c806e6a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f07d80011e0 tx=0x7f07d8007d30 comp rx=0 tx=0).stop 2023-10-06T14:26:26.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 10 monclient: shutdown 2023-10-06T14:26:26.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:26.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 1 -- 172.21.15.19:0/2529242516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07e40b3990 msgr2=0x7f07e40a9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:26.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 1 --2- 172.21.15.19:0/2529242516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07e40b3990 0x7f07e40a9970 secure :-1 s=READY pgs=5105 cs=0 l=1 rev1=1 crypto rx=0x7f07e4140f10 tx=0x7f07e00046c0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 1 -- 172.21.15.19:0/2529242516 shutdown_connections 2023-10-06T14:26:26.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 1 --2- 172.21.15.19:0/2529242516 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f07c806c1f0 0x7f07c806e6a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 1 --2- 172.21.15.19:0/2529242516 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f07e40b3d60 0x7f07e40a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 1 --2- 172.21.15.19:0/2529242516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07e40b3990 0x7f07e40a9970 unknown :-1 s=CLOSED pgs=5105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 1 --2- 172.21.15.19:0/2529242516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40b00c0 0x7f07e413d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 1 -- 172.21.15.19:0/2529242516 >> 172.21.15.19:0/2529242516 conn(0x7f07e40b6e00 msgr2=0x7f07e413a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:26.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 1 -- 172.21.15.19:0/2529242516 shutdown_connections 2023-10-06T14:26:26.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 1 -- 172.21.15.19:0/2529242516 wait complete. 2023-10-06T14:26:26.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.276+0000 7f07eb4b6640 1 librados: shutdown 2023-10-06T14:26:26.289 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":1972,"stamp":"2023-10-06T14:26:25.803802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":514760,"kb_used_data":44936,"kb_used_omap":0,"kb_used_meta":469760,"kb_avail":754459960,"statfs":{"total":773094113280,"available":772566999040,"internally_reserved":0,"allocated":46014464,"data_stored":24890348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":481034240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":406,"apply_latency_ms":406,"commit_latency_ns":406000000,"apply_latency_ns":406000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4000299,"num_objects":-44,"num_object_clones":-1,"num_object_copies":-132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44,"num_whiteouts":0,"num_read":-31,"num_read_kb":-11,"num_write":-71,"num_write_kb":-3969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.114550"},"pg_stats":[{"pgid":"8.14","version":"0'0","reported_seq":921,"reported_epoch":966,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.757496+0000","last_change":"2023-10-06T13:54:27.674098+0000","last_active":"2023-10-06T14:26:21.757496+0000","last_peered":"2023-10-06T14:26:21.757496+0000","last_clean":"2023-10-06T14:26:21.757496+0000","last_became_active":"2023-10-06T13:54:27.664679+0000","last_became_peered":"2023-10-06T13:54:27.664679+0000","last_unstale":"2023-10-06T14:26:21.757496+0000","last_undegraded":"2023-10-06T14:26:21.757496+0000","last_fullsized":"2023-10-06T14:26:21.757496+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:57:24.020996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":920,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.836623+0000","last_change":"2023-10-06T14:22:39.751625+0000","last_active":"2023-10-06T14:26:22.836623+0000","last_peered":"2023-10-06T14:26:22.836623+0000","last_clean":"2023-10-06T14:26:22.836623+0000","last_became_active":"2023-10-06T14:22:39.724017+0000","last_became_peered":"2023-10-06T14:22:39.724017+0000","last_unstale":"2023-10-06T14:26:22.836623+0000","last_undegraded":"2023-10-06T14:26:22.836623+0000","last_fullsized":"2023-10-06T14:26:22.836623+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:31.357267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":924,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.734022+0000","last_change":"2023-10-06T14:16:33.556535+0000","last_active":"2023-10-06T14:26:20.734022+0000","last_peered":"2023-10-06T14:26:20.734022+0000","last_clean":"2023-10-06T14:26:20.734022+0000","last_became_active":"2023-10-06T14:16:33.542923+0000","last_became_peered":"2023-10-06T14:16:33.542923+0000","last_unstale":"2023-10-06T14:26:20.734022+0000","last_undegraded":"2023-10-06T14:26:20.734022+0000","last_fullsized":"2023-10-06T14:26:20.734022+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:56:19.999321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":927,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.813484+0000","last_change":"2023-10-06T13:54:29.208698+0000","last_active":"2023-10-06T14:26:22.813484+0000","last_peered":"2023-10-06T14:26:22.813484+0000","last_clean":"2023-10-06T14:26:22.813484+0000","last_became_active":"2023-10-06T13:54:29.163161+0000","last_became_peered":"2023-10-06T13:54:29.163161+0000","last_unstale":"2023-10-06T14:26:22.813484+0000","last_undegraded":"2023-10-06T14:26:22.813484+0000","last_fullsized":"2023-10-06T14:26:22.813484+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:22.518227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":919,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.795205+0000","last_change":"2023-10-06T14:22:39.220927+0000","last_active":"2023-10-06T14:26:22.795205+0000","last_peered":"2023-10-06T14:26:22.795205+0000","last_clean":"2023-10-06T14:26:22.795205+0000","last_became_active":"2023-10-06T14:22:39.128927+0000","last_became_peered":"2023-10-06T14:22:39.128927+0000","last_unstale":"2023-10-06T14:26:22.795205+0000","last_undegraded":"2023-10-06T14:26:22.795205+0000","last_fullsized":"2023-10-06T14:26:22.795205+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:02.237203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":920,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.885756+0000","last_change":"2023-10-06T14:22:39.468045+0000","last_active":"2023-10-06T14:26:22.885756+0000","last_peered":"2023-10-06T14:26:22.885756+0000","last_clean":"2023-10-06T14:26:22.885756+0000","last_became_active":"2023-10-06T14:22:39.418221+0000","last_became_peered":"2023-10-06T14:22:39.418221+0000","last_unstale":"2023-10-06T14:26:22.885756+0000","last_undegraded":"2023-10-06T14:26:22.885756+0000","last_fullsized":"2023-10-06T14:26:22.885756+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:46.111461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":920,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.866951+0000","last_change":"2023-10-06T14:22:39.210447+0000","last_active":"2023-10-06T14:26:22.866951+0000","last_peered":"2023-10-06T14:26:22.866951+0000","last_clean":"2023-10-06T14:26:22.866951+0000","last_became_active":"2023-10-06T14:22:39.195040+0000","last_became_peered":"2023-10-06T14:22:39.195040+0000","last_unstale":"2023-10-06T14:26:22.866951+0000","last_undegraded":"2023-10-06T14:26:22.866951+0000","last_fullsized":"2023-10-06T14:26:22.866951+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:29:47.926237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":925,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.729122+0000","last_change":"2023-10-06T14:16:33.305137+0000","last_active":"2023-10-06T14:26:20.729122+0000","last_peered":"2023-10-06T14:26:20.729122+0000","last_clean":"2023-10-06T14:26:20.729122+0000","last_became_active":"2023-10-06T14:16:33.300222+0000","last_became_peered":"2023-10-06T14:16:33.300222+0000","last_unstale":"2023-10-06T14:26:20.729122+0000","last_undegraded":"2023-10-06T14:26:20.729122+0000","last_fullsized":"2023-10-06T14:26:20.729122+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:34:52.750381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":919,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.822743+0000","last_change":"2023-10-06T14:22:39.745594+0000","last_active":"2023-10-06T14:26:22.822743+0000","last_peered":"2023-10-06T14:26:22.822743+0000","last_clean":"2023-10-06T14:26:22.822743+0000","last_became_active":"2023-10-06T14:22:39.712413+0000","last_became_peered":"2023-10-06T14:22:39.712413+0000","last_unstale":"2023-10-06T14:26:22.822743+0000","last_undegraded":"2023-10-06T14:26:22.822743+0000","last_fullsized":"2023-10-06T14:26:22.822743+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:59:53.774197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":905,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.206280+0000","last_change":"2023-10-06T13:54:19.475064+0000","last_active":"2023-10-06T14:26:21.206280+0000","last_peered":"2023-10-06T14:26:21.206280+0000","last_clean":"2023-10-06T14:26:21.206280+0000","last_became_active":"2023-10-06T13:54:19.427005+0000","last_became_peered":"2023-10-06T13:54:19.427005+0000","last_unstale":"2023-10-06T14:26:21.206280+0000","last_undegraded":"2023-10-06T14:26:21.206280+0000","last_fullsized":"2023-10-06T14:26:21.206280+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:21.985890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":910,"reported_epoch":968,"state":"active+clean","last_fresh":"2023-10-06T14:26:23.397157+0000","last_change":"2023-10-06T13:54:19.536231+0000","last_active":"2023-10-06T14:26:23.397157+0000","last_peered":"2023-10-06T14:26:23.397157+0000","last_clean":"2023-10-06T14:26:23.397157+0000","last_became_active":"2023-10-06T13:54:19.531077+0000","last_became_peered":"2023-10-06T13:54:19.531077+0000","last_unstale":"2023-10-06T14:26:23.397157+0000","last_undegraded":"2023-10-06T14:26:23.397157+0000","last_fullsized":"2023-10-06T14:26:23.397157+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:05.864633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":917,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.607069+0000","last_change":"2023-10-06T14:06:28.635986+0000","last_active":"2023-10-06T14:26:19.607069+0000","last_peered":"2023-10-06T14:26:19.607069+0000","last_clean":"2023-10-06T14:26:19.607069+0000","last_became_active":"2023-10-06T14:06:28.630618+0000","last_became_peered":"2023-10-06T14:06:28.630618+0000","last_unstale":"2023-10-06T14:26:19.607069+0000","last_undegraded":"2023-10-06T14:26:19.607069+0000","last_fullsized":"2023-10-06T14:26:19.607069+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:32.474746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":927,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.694758+0000","last_change":"2023-10-06T13:54:21.830131+0000","last_active":"2023-10-06T14:26:19.694758+0000","last_peered":"2023-10-06T14:26:19.694758+0000","last_clean":"2023-10-06T14:26:19.694758+0000","last_became_active":"2023-10-06T13:54:21.813915+0000","last_became_peered":"2023-10-06T13:54:21.813915+0000","last_unstale":"2023-10-06T14:26:19.694758+0000","last_undegraded":"2023-10-06T14:26:19.694758+0000","last_fullsized":"2023-10-06T14:26:19.694758+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:21:55.469659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":922,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.832923+0000","last_change":"2023-10-06T14:22:39.679043+0000","last_active":"2023-10-06T14:26:22.832923+0000","last_peered":"2023-10-06T14:26:22.832923+0000","last_clean":"2023-10-06T14:26:22.832923+0000","last_became_active":"2023-10-06T14:22:39.673285+0000","last_became_peered":"2023-10-06T14:22:39.673285+0000","last_unstale":"2023-10-06T14:26:22.832923+0000","last_undegraded":"2023-10-06T14:26:22.832923+0000","last_fullsized":"2023-10-06T14:26:22.832923+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:12.259360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":913,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.740540+0000","last_change":"2023-10-06T14:16:33.403287+0000","last_active":"2023-10-06T14:26:20.740540+0000","last_peered":"2023-10-06T14:26:20.740540+0000","last_clean":"2023-10-06T14:26:20.740540+0000","last_became_active":"2023-10-06T14:16:33.392050+0000","last_became_peered":"2023-10-06T14:16:33.392050+0000","last_unstale":"2023-10-06T14:26:20.740540+0000","last_undegraded":"2023-10-06T14:26:20.740540+0000","last_fullsized":"2023-10-06T14:26:20.740540+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:29:10.779601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":920,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.617510+0000","last_change":"2023-10-06T14:05:13.253436+0000","last_active":"2023-10-06T14:26:19.617510+0000","last_peered":"2023-10-06T14:26:19.617510+0000","last_clean":"2023-10-06T14:26:19.617510+0000","last_became_active":"2023-10-06T14:05:13.248166+0000","last_became_peered":"2023-10-06T14:05:13.248166+0000","last_unstale":"2023-10-06T14:26:19.617510+0000","last_undegraded":"2023-10-06T14:26:19.617510+0000","last_fullsized":"2023-10-06T14:26:19.617510+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:51:29.023023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":908,"reported_epoch":968,"state":"active+clean","last_fresh":"2023-10-06T14:26:23.401249+0000","last_change":"2023-10-06T13:54:26.224788+0000","last_active":"2023-10-06T14:26:23.401249+0000","last_peered":"2023-10-06T14:26:23.401249+0000","last_clean":"2023-10-06T14:26:23.401249+0000","last_became_active":"2023-10-06T13:54:26.218851+0000","last_became_peered":"2023-10-06T13:54:26.218851+0000","last_unstale":"2023-10-06T14:26:23.401249+0000","last_undegraded":"2023-10-06T14:26:23.401249+0000","last_fullsized":"2023-10-06T14:26:23.401249+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:43:20.812836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":939,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.838339+0000","last_change":"2023-10-06T14:22:39.234795+0000","last_active":"2023-10-06T14:26:22.838339+0000","last_peered":"2023-10-06T14:26:22.838339+0000","last_clean":"2023-10-06T14:26:22.838339+0000","last_became_active":"2023-10-06T14:22:39.109893+0000","last_became_peered":"2023-10-06T14:22:39.109893+0000","last_unstale":"2023-10-06T14:26:22.838339+0000","last_undegraded":"2023-10-06T14:26:22.838339+0000","last_fullsized":"2023-10-06T14:26:22.838339+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:27.039844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":925,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.735476+0000","last_change":"2023-10-06T14:16:33.559587+0000","last_active":"2023-10-06T14:26:20.735476+0000","last_peered":"2023-10-06T14:26:20.735476+0000","last_clean":"2023-10-06T14:26:20.735476+0000","last_became_active":"2023-10-06T14:16:33.551586+0000","last_became_peered":"2023-10-06T14:16:33.551586+0000","last_unstale":"2023-10-06T14:26:20.735476+0000","last_undegraded":"2023-10-06T14:26:20.735476+0000","last_fullsized":"2023-10-06T14:26:20.735476+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:53.251968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":918,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.741987+0000","last_change":"2023-10-06T14:16:33.406336+0000","last_active":"2023-10-06T14:26:20.741987+0000","last_peered":"2023-10-06T14:26:20.741987+0000","last_clean":"2023-10-06T14:26:20.741987+0000","last_became_active":"2023-10-06T14:16:33.398349+0000","last_became_peered":"2023-10-06T14:16:33.398349+0000","last_unstale":"2023-10-06T14:26:20.741987+0000","last_undegraded":"2023-10-06T14:26:20.741987+0000","last_fullsized":"2023-10-06T14:26:20.741987+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:24:58.230306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":928,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.738574+0000","last_change":"2023-10-06T13:54:31.926742+0000","last_active":"2023-10-06T14:26:22.738574+0000","last_peered":"2023-10-06T14:26:22.738574+0000","last_clean":"2023-10-06T14:26:22.738574+0000","last_became_active":"2023-10-06T13:54:31.908571+0000","last_became_peered":"2023-10-06T13:54:31.908571+0000","last_unstale":"2023-10-06T14:26:22.738574+0000","last_undegraded":"2023-10-06T14:26:22.738574+0000","last_fullsized":"2023-10-06T14:26:22.738574+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:45.581588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":928,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.868651+0000","last_change":"2023-10-06T14:22:39.277750+0000","last_active":"2023-10-06T14:26:22.868651+0000","last_peered":"2023-10-06T14:26:22.868651+0000","last_clean":"2023-10-06T14:26:22.868651+0000","last_became_active":"2023-10-06T14:22:39.078831+0000","last_became_peered":"2023-10-06T14:22:39.078831+0000","last_unstale":"2023-10-06T14:26:22.868651+0000","last_undegraded":"2023-10-06T14:26:22.868651+0000","last_fullsized":"2023-10-06T14:26:22.868651+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:52.651981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":916,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.595315+0000","last_change":"2023-10-06T14:05:13.235606+0000","last_active":"2023-10-06T14:26:19.595315+0000","last_peered":"2023-10-06T14:26:19.595315+0000","last_clean":"2023-10-06T14:26:19.595315+0000","last_became_active":"2023-10-06T14:05:13.230098+0000","last_became_peered":"2023-10-06T14:05:13.230098+0000","last_unstale":"2023-10-06T14:26:19.595315+0000","last_undegraded":"2023-10-06T14:26:19.595315+0000","last_fullsized":"2023-10-06T14:26:19.595315+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:28:31.276617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":928,"reported_epoch":968,"state":"active+clean","last_fresh":"2023-10-06T14:26:23.379085+0000","last_change":"2023-10-06T14:16:33.641985+0000","last_active":"2023-10-06T14:26:23.379085+0000","last_peered":"2023-10-06T14:26:23.379085+0000","last_clean":"2023-10-06T14:26:23.379085+0000","last_became_active":"2023-10-06T14:16:33.635681+0000","last_became_peered":"2023-10-06T14:16:33.635681+0000","last_unstale":"2023-10-06T14:26:23.379085+0000","last_undegraded":"2023-10-06T14:26:23.379085+0000","last_fullsized":"2023-10-06T14:26:23.379085+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:42.455510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":927,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.720626+0000","last_change":"2023-10-06T14:16:33.275781+0000","last_active":"2023-10-06T14:26:20.720626+0000","last_peered":"2023-10-06T14:26:20.720626+0000","last_clean":"2023-10-06T14:26:20.720626+0000","last_became_active":"2023-10-06T14:16:33.269648+0000","last_became_peered":"2023-10-06T14:16:33.269648+0000","last_unstale":"2023-10-06T14:26:20.720626+0000","last_undegraded":"2023-10-06T14:26:20.720626+0000","last_fullsized":"2023-10-06T14:26:20.720626+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:25.574583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":938,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.797168+0000","last_change":"2023-10-06T14:22:39.170180+0000","last_active":"2023-10-06T14:26:22.797168+0000","last_peered":"2023-10-06T14:26:22.797168+0000","last_clean":"2023-10-06T14:26:22.797168+0000","last_became_active":"2023-10-06T14:22:39.121176+0000","last_became_peered":"2023-10-06T14:22:39.121176+0000","last_unstale":"2023-10-06T14:26:22.797168+0000","last_undegraded":"2023-10-06T14:26:22.797168+0000","last_fullsized":"2023-10-06T14:26:22.797168+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:24.770331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":922,"reported_epoch":966,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.674582+0000","last_change":"2023-10-06T13:54:27.649937+0000","last_active":"2023-10-06T14:26:21.674582+0000","last_peered":"2023-10-06T14:26:21.674582+0000","last_clean":"2023-10-06T14:26:21.674582+0000","last_became_active":"2023-10-06T13:54:27.642873+0000","last_became_peered":"2023-10-06T13:54:27.642873+0000","last_unstale":"2023-10-06T14:26:21.674582+0000","last_undegraded":"2023-10-06T14:26:21.674582+0000","last_fullsized":"2023-10-06T14:26:21.674582+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:36:36.269635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":926,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.805049+0000","last_change":"2023-10-06T14:22:39.756236+0000","last_active":"2023-10-06T14:26:22.805049+0000","last_peered":"2023-10-06T14:26:22.805049+0000","last_clean":"2023-10-06T14:26:22.805049+0000","last_became_active":"2023-10-06T14:22:39.733473+0000","last_became_peered":"2023-10-06T14:22:39.733473+0000","last_unstale":"2023-10-06T14:26:22.805049+0000","last_undegraded":"2023-10-06T14:26:22.805049+0000","last_fullsized":"2023-10-06T14:26:22.805049+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:09.280243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":1053,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.740919+0000","last_change":"2023-10-06T14:16:33.639507+0000","last_active":"2023-10-06T14:26:22.740919+0000","last_peered":"2023-10-06T14:26:22.740919+0000","last_clean":"2023-10-06T14:26:22.740919+0000","last_became_active":"2023-10-06T14:16:33.632776+0000","last_became_peered":"2023-10-06T14:16:33.632776+0000","last_unstale":"2023-10-06T14:26:22.740919+0000","last_undegraded":"2023-10-06T14:26:22.740919+0000","last_fullsized":"2023-10-06T14:26:22.740919+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:08.156870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:08.156870+0000","last_clean_scrub_stamp":"2023-10-06T13:53:08.156870+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:08.683941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":908,"reported_epoch":968,"state":"active+clean","last_fresh":"2023-10-06T14:26:23.385649+0000","last_change":"2023-10-06T13:54:26.199772+0000","last_active":"2023-10-06T14:26:23.385649+0000","last_peered":"2023-10-06T14:26:23.385649+0000","last_clean":"2023-10-06T14:26:23.385649+0000","last_became_active":"2023-10-06T13:54:26.193354+0000","last_became_peered":"2023-10-06T13:54:26.193354+0000","last_unstale":"2023-10-06T14:26:23.385649+0000","last_undegraded":"2023-10-06T14:26:23.385649+0000","last_fullsized":"2023-10-06T14:26:23.385649+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:11.930838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":928,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.609432+0000","last_change":"2023-10-06T13:54:22.015064+0000","last_active":"2023-10-06T14:26:19.609432+0000","last_peered":"2023-10-06T14:26:19.609432+0000","last_clean":"2023-10-06T14:26:19.609432+0000","last_became_active":"2023-10-06T13:54:22.005243+0000","last_became_peered":"2023-10-06T13:54:22.005243+0000","last_unstale":"2023-10-06T14:26:19.609432+0000","last_undegraded":"2023-10-06T14:26:19.609432+0000","last_fullsized":"2023-10-06T14:26:19.609432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:46:20.602620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"43'7","reported_seq":935,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.193835+0000","last_change":"2023-10-06T14:05:13.464997+0000","last_active":"2023-10-06T14:26:21.193835+0000","last_peered":"2023-10-06T14:26:21.193835+0000","last_clean":"2023-10-06T14:26:21.193835+0000","last_became_active":"2023-10-06T14:05:13.458219+0000","last_became_peered":"2023-10-06T14:05:13.458219+0000","last_unstale":"2023-10-06T14:26:21.193835+0000","last_undegraded":"2023-10-06T14:26:21.193835+0000","last_fullsized":"2023-10-06T14:26:21.193835+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:40:26.823980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":928,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.694527+0000","last_change":"2023-10-06T14:22:39.278351+0000","last_active":"2023-10-06T14:26:20.694527+0000","last_peered":"2023-10-06T14:26:20.694527+0000","last_clean":"2023-10-06T14:26:20.694527+0000","last_became_active":"2023-10-06T14:22:39.269435+0000","last_became_peered":"2023-10-06T14:22:39.269435+0000","last_unstale":"2023-10-06T14:26:20.694527+0000","last_undegraded":"2023-10-06T14:26:20.694527+0000","last_fullsized":"2023-10-06T14:26:20.694527+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:10.817190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":921,"reported_epoch":968,"state":"active+clean","last_fresh":"2023-10-06T14:26:23.402751+0000","last_change":"2023-10-06T14:16:33.659581+0000","last_active":"2023-10-06T14:26:23.402751+0000","last_peered":"2023-10-06T14:26:23.402751+0000","last_clean":"2023-10-06T14:26:23.402751+0000","last_became_active":"2023-10-06T14:16:33.650127+0000","last_became_peered":"2023-10-06T14:16:33.650127+0000","last_unstale":"2023-10-06T14:26:23.402751+0000","last_undegraded":"2023-10-06T14:26:23.402751+0000","last_fullsized":"2023-10-06T14:26:23.402751+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:45.333800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":907,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.213411+0000","last_change":"2023-10-06T13:54:22.165757+0000","last_active":"2023-10-06T14:26:21.213411+0000","last_peered":"2023-10-06T14:26:21.213411+0000","last_clean":"2023-10-06T14:26:21.213411+0000","last_became_active":"2023-10-06T13:54:22.160421+0000","last_became_peered":"2023-10-06T13:54:22.160421+0000","last_unstale":"2023-10-06T14:26:21.213411+0000","last_undegraded":"2023-10-06T14:26:21.213411+0000","last_fullsized":"2023-10-06T14:26:21.213411+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:51.552552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":929,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.790789+0000","last_change":"2023-10-06T13:54:25.399248+0000","last_active":"2023-10-06T14:26:22.790789+0000","last_peered":"2023-10-06T14:26:22.790789+0000","last_clean":"2023-10-06T14:26:22.790789+0000","last_became_active":"2023-10-06T13:54:25.349894+0000","last_became_peered":"2023-10-06T13:54:25.349894+0000","last_unstale":"2023-10-06T14:26:22.790789+0000","last_undegraded":"2023-10-06T14:26:22.790789+0000","last_fullsized":"2023-10-06T14:26:22.790789+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:12:02.751197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":941,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.684113+0000","last_change":"2023-10-06T14:22:39.984989+0000","last_active":"2023-10-06T14:26:19.684113+0000","last_peered":"2023-10-06T14:26:19.684113+0000","last_clean":"2023-10-06T14:26:19.684113+0000","last_became_active":"2023-10-06T14:22:39.979167+0000","last_became_peered":"2023-10-06T14:22:39.979167+0000","last_unstale":"2023-10-06T14:26:19.684113+0000","last_undegraded":"2023-10-06T14:26:19.684113+0000","last_fullsized":"2023-10-06T14:26:19.684113+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":836,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:55:13.743000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":937,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.635697+0000","last_change":"2023-10-06T14:05:13.200912+0000","last_active":"2023-10-06T14:26:19.635697+0000","last_peered":"2023-10-06T14:26:19.635697+0000","last_clean":"2023-10-06T14:26:19.635697+0000","last_became_active":"2023-10-06T14:05:13.191757+0000","last_became_peered":"2023-10-06T14:05:13.191757+0000","last_unstale":"2023-10-06T14:26:19.635697+0000","last_undegraded":"2023-10-06T14:26:19.635697+0000","last_fullsized":"2023-10-06T14:26:19.635697+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:21:31.319110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":906,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.187963+0000","last_change":"2023-10-06T13:54:22.153461+0000","last_active":"2023-10-06T14:26:21.187963+0000","last_peered":"2023-10-06T14:26:21.187963+0000","last_clean":"2023-10-06T14:26:21.187963+0000","last_became_active":"2023-10-06T13:54:22.144576+0000","last_became_peered":"2023-10-06T13:54:22.144576+0000","last_unstale":"2023-10-06T14:26:21.187963+0000","last_undegraded":"2023-10-06T14:26:21.187963+0000","last_fullsized":"2023-10-06T14:26:21.187963+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:55:12.616531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":927,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.763619+0000","last_change":"2023-10-06T13:54:19.740406+0000","last_active":"2023-10-06T14:26:22.763619+0000","last_peered":"2023-10-06T14:26:22.763619+0000","last_clean":"2023-10-06T14:26:22.763619+0000","last_became_active":"2023-10-06T13:54:19.734381+0000","last_became_peered":"2023-10-06T13:54:19.734381+0000","last_unstale":"2023-10-06T14:26:22.763619+0000","last_undegraded":"2023-10-06T14:26:22.763619+0000","last_fullsized":"2023-10-06T14:26:22.763619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:23.226096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":934,"reported_epoch":966,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.652071+0000","last_change":"2023-10-06T14:05:13.451797+0000","last_active":"2023-10-06T14:26:21.652071+0000","last_peered":"2023-10-06T14:26:21.652071+0000","last_clean":"2023-10-06T14:26:21.652071+0000","last_became_active":"2023-10-06T14:05:13.442612+0000","last_became_peered":"2023-10-06T14:05:13.442612+0000","last_unstale":"2023-10-06T14:26:21.652071+0000","last_undegraded":"2023-10-06T14:26:21.652071+0000","last_fullsized":"2023-10-06T14:26:21.652071+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:59:33.582696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719477122,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59108,"kb_used_data":5916,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312732,"statfs":{"total":96636764160,"available":96576237568,"internally_reserved":0,"allocated":6057984,"data_stored":3404017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":11.946,"5min":12.488,"15min":13.279},"min":{"1min":6.3920000000000003,"5min":4.6070000000000002,"15min":4.6070000000000002},"max":{"1min":52.079999999999998,"5min":91.744,"15min":109.691},"last":9.3650000000000002},{"interface":"front","average":{"1min":12.875999999999999,"5min":12.664,"15min":14.576000000000001},"min":{"1min":4.1870000000000003,"5min":4.1870000000000003,"15min":4.1870000000000003},"max":{"1min":44.232999999999997,"5min":67.936000000000007,"15min":166.33699999999999},"last":12.683999999999999}]},{"osd":2,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":12.112,"5min":11.595000000000001,"15min":13.598000000000001},"min":{"1min":7.9850000000000003,"5min":6.3920000000000003,"15min":4.9800000000000004},"max":{"1min":18.631,"5min":64.661000000000001,"15min":155.68799999999999},"last":24.128},{"interface":"front","average":{"1min":15.757999999999999,"5min":13.82,"15min":16.335000000000001},"min":{"1min":9.0660000000000007,"5min":7.3049999999999997,"15min":6.6059999999999999},"max":{"1min":47.368000000000002,"5min":92.335999999999999,"15min":212.16399999999999},"last":18.623999999999999}]},{"osd":3,"last update":"Fri Oct 6 14:25:44 2023","interfaces":[{"interface":"back","average":{"1min":23.273,"5min":29.661000000000001,"15min":33.582999999999998},"min":{"1min":9.6639999999999997,"5min":8.6850000000000005,"15min":8.6850000000000005},"max":{"1min":126.22799999999999,"5min":310.19,"15min":310.19},"last":13.875},{"interface":"front","average":{"1min":18.346,"5min":18.327999999999999,"15min":18.873999999999999},"min":{"1min":8.5370000000000008,"5min":7.8399999999999999,"15min":7.8399999999999999},"max":{"1min":65.375,"5min":65.375,"15min":65.375},"last":7.8600000000000003}]},{"osd":4,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":15290.522000000001,"5min":3070.721,"15min":1038.3879999999999},"min":{"1min":9.2720000000000002,"5min":7.8520000000000003,"15min":7.0190000000000001},"max":{"1min":39693.332999999999,"5min":39693.332999999999,"15min":39693.332999999999},"last":13.286},{"interface":"front","average":{"1min":15323.151,"5min":3078.1880000000001,"15min":1039.8889999999999},"min":{"1min":14.579000000000001,"5min":8.5679999999999996,"15min":7.3239999999999998},"max":{"1min":39697.402000000002,"5min":39697.402000000002,"15min":39697.402000000002},"last":11.196}]},{"osd":5,"last update":"Fri Oct 6 14:25:44 2023","interfaces":[{"interface":"back","average":{"1min":21.614999999999998,"5min":17.170000000000002,"15min":92.123000000000005},"min":{"1min":9.1199999999999992,"5min":9.1199999999999992,"15min":9.1199999999999992},"max":{"1min":69.561999999999998,"5min":69.561999999999998,"15min":8890.4069999999992},"last":17.754999999999999},{"interface":"front","average":{"1min":25.356000000000002,"5min":17.224,"15min":92.638999999999996},"min":{"1min":8.3140000000000001,"5min":7.8620000000000001,"15min":7.8620000000000001},"max":{"1min":139.78800000000001,"5min":139.78800000000001,"15min":8888.7939999999999},"last":14.494}]},{"osd":6,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.563000000000001,"5min":15.667999999999999,"15min":1233.97},"min":{"1min":9.9359999999999999,"5min":8.4710000000000001,"15min":7.3529999999999998},"max":{"1min":20.318000000000001,"5min":78.221999999999994,"15min":47304.226999999999},"last":20.283000000000001},{"interface":"front","average":{"1min":17.091000000000001,"5min":18.989000000000001,"15min":1237.328},"min":{"1min":11.202999999999999,"5min":10.666,"15min":8.9949999999999992},"max":{"1min":24.579000000000001,"5min":116.84999999999999,"15min":47302.955999999998},"last":16.690999999999999}]},{"osd":7,"last update":"Fri Oct 6 14:25:31 2023","interfaces":[{"interface":"back","average":{"1min":26.558,"5min":20.433,"15min":22.890000000000001},"min":{"1min":14.305999999999999,"5min":10.992000000000001,"15min":9.0250000000000004},"max":{"1min":78.769000000000005,"5min":78.769000000000005,"15min":214.02699999999999},"last":21.684999999999999},{"interface":"front","average":{"1min":30.707000000000001,"5min":20.494,"15min":22.446999999999999},"min":{"1min":11.108000000000001,"5min":9.8379999999999992,"15min":9.6159999999999997},"max":{"1min":134.17400000000001,"5min":134.17400000000001,"15min":201.52799999999999},"last":11.955}]}]},{"osd":0,"up_from":16,"seq":68719477138,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58716,"kb_used_data":5524,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313124,"statfs":{"total":96636764160,"available":96576638976,"internally_reserved":0,"allocated":5656576,"data_stored":3006177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":53,"apply_latency_ms":53,"commit_latency_ns":53000000,"apply_latency_ns":53000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 14:26:15 2023","interfaces":[{"interface":"back","average":{"1min":19.928999999999998,"5min":17.440999999999999,"15min":16.654},"min":{"1min":6.7800000000000002,"5min":5.4980000000000002,"15min":4.9720000000000004},"max":{"1min":97.855999999999995,"5min":135.13800000000001,"15min":342.61399999999998},"last":11.712},{"interface":"front","average":{"1min":21.222000000000001,"5min":19.111999999999998,"15min":17.628},"min":{"1min":5.9409999999999998,"5min":5.8810000000000002,"15min":5.5960000000000001},"max":{"1min":144.221,"5min":144.221,"15min":337.48000000000002},"last":13.032}]},{"osd":2,"last update":"Fri Oct 6 14:25:21 2023","interfaces":[{"interface":"back","average":{"1min":34.204999999999998,"5min":19.474,"15min":19.213000000000001},"min":{"1min":8.8699999999999992,"5min":7.2809999999999997,"15min":5.9669999999999996},"max":{"1min":184.036,"5min":184.036,"15min":184.036},"last":11.177},{"interface":"front","average":{"1min":30.013999999999999,"5min":15.800000000000001,"15min":17.484999999999999},"min":{"1min":5.718,"5min":5.718,"15min":5.718},"max":{"1min":181.06700000000001,"5min":181.06700000000001,"15min":181.06700000000001},"last":6.6470000000000002}]},{"osd":3,"last update":"Fri Oct 6 14:25:42 2023","interfaces":[{"interface":"back","average":{"1min":30.332000000000001,"5min":24.734999999999999,"15min":25.606999999999999},"min":{"1min":7.5949999999999998,"5min":7.4139999999999997,"15min":7.4139999999999997},"max":{"1min":189.10499999999999,"5min":189.10499999999999,"15min":189.10499999999999},"last":13.619999999999999},{"interface":"front","average":{"1min":31.155000000000001,"5min":24.603999999999999,"15min":24.998000000000001},"min":{"1min":10.960000000000001,"5min":6.9850000000000003,"15min":6.9850000000000003},"max":{"1min":163.60900000000001,"5min":163.60900000000001,"15min":163.60900000000001},"last":16.693999999999999}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":3655.643,"5min":746.56799999999998,"15min":262.995},"min":{"1min":13.273999999999999,"5min":8.7420000000000009,"15min":6.9269999999999996},"max":{"1min":39913.453999999998,"5min":39913.453999999998,"15min":39913.453999999998},"last":15.192},{"interface":"front","average":{"1min":3644.9029999999998,"5min":742.30499999999995,"15min":261.23700000000002},"min":{"1min":7.2050000000000001,"5min":7.2050000000000001,"15min":7.2050000000000001},"max":{"1min":39912.421999999999,"5min":39912.421999999999,"15min":39912.421999999999},"last":9.0779999999999994}]},{"osd":5,"last update":"Fri Oct 6 14:25:49 2023","interfaces":[{"interface":"back","average":{"1min":15.067,"5min":16.125,"15min":71.585999999999999},"min":{"1min":8.2599999999999998,"5min":7.5209999999999999,"15min":7.5209999999999999},"max":{"1min":56.817,"5min":142.267,"15min":6380.991},"last":17.486000000000001},{"interface":"front","average":{"1min":24.161000000000001,"5min":20.405000000000001,"15min":66.221000000000004},"min":{"1min":10.645,"5min":9.9390000000000001,"15min":9.4550000000000001},"max":{"1min":133.797,"5min":145.59299999999999,"15min":6266.0159999999996},"last":12.458}]},{"osd":6,"last update":"Fri Oct 6 14:26:05 2023","interfaces":[{"interface":"back","average":{"1min":13.15,"5min":20.032,"15min":725.85500000000002},"min":{"1min":7.7789999999999999,"5min":7.1479999999999997,"15min":7.1479999999999997},"max":{"1min":26.087,"5min":177.73699999999999,"15min":46416.196000000004},"last":10.26},{"interface":"front","average":{"1min":15.784000000000001,"5min":22.670000000000002,"15min":724.05899999999997},"min":{"1min":10.669,"5min":8.6609999999999996,"15min":6.9459999999999997},"max":{"1min":21.206,"5min":181.625,"15min":46403.283000000003},"last":18.282}]},{"osd":7,"last update":"Fri Oct 6 14:25:31 2023","interfaces":[{"interface":"back","average":{"1min":38.594000000000001,"5min":23.141999999999999,"15min":1331.846},"min":{"1min":14.388,"5min":10.557,"15min":9.8209999999999997},"max":{"1min":178.86500000000001,"5min":178.86500000000001,"15min":67028.107999999993},"last":14.598000000000001},{"interface":"front","average":{"1min":31.856999999999999,"5min":20.099,"15min":1329.4469999999999},"min":{"1min":9.3140000000000001,"5min":8.4320000000000004,"15min":8.4320000000000004},"max":{"1min":178.38900000000001,"5min":178.38900000000001,"15min":67001.737999999998},"last":9.6379999999999999}]}]},{"osd":4,"up_from":430,"seq":1846835937654,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58704,"kb_used_data":5512,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313136,"statfs":{"total":96636764160,"available":96576651264,"internally_reserved":0,"allocated":5644288,"data_stored":3017805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":61.039999999999999,"5min":21.318999999999999,"15min":15.34},"min":{"1min":7.0979999999999999,"5min":4.508,"15min":4.085},"max":{"1min":469.096,"5min":469.096,"15min":469.096},"last":7.2140000000000004},{"interface":"front","average":{"1min":60.287999999999997,"5min":20.288,"15min":17.068999999999999},"min":{"1min":8.5890000000000004,"5min":4.758,"15min":4.5999999999999996},"max":{"1min":423.42000000000002,"5min":423.42000000000002,"15min":423.42000000000002},"last":7.75}]},{"osd":1,"last update":"Fri Oct 6 14:26:13 2023","interfaces":[{"interface":"back","average":{"1min":81.888000000000005,"5min":27.257000000000001,"15min":19.550000000000001},"min":{"1min":7.3289999999999997,"5min":5.9850000000000003,"15min":4.9390000000000001},"max":{"1min":469.72800000000001,"5min":469.72800000000001,"15min":469.72800000000001},"last":10.218999999999999},{"interface":"front","average":{"1min":77.108999999999995,"5min":26.469999999999999,"15min":21.765999999999998},"min":{"1min":5.9320000000000004,"5min":5.9320000000000004,"15min":5.9320000000000004},"max":{"1min":425.584,"5min":425.584,"15min":425.584},"last":10.827999999999999}]},{"osd":2,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":113.371,"5min":34.213000000000001,"15min":23.704999999999998},"min":{"1min":7.6429999999999998,"5min":6.8159999999999998,"15min":6.4770000000000003},"max":{"1min":530.30399999999997,"5min":530.30399999999997,"15min":530.30399999999997},"last":12.532999999999999},{"interface":"front","average":{"1min":117.30200000000001,"5min":37.220999999999997,"15min":26.550999999999998},"min":{"1min":10.667999999999999,"5min":7.0750000000000002,"15min":6.5780000000000003},"max":{"1min":530.86099999999999,"5min":530.86099999999999,"15min":530.86099999999999},"last":13.74}]},{"osd":3,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":58.695,"5min":25.254999999999999,"15min":19.239000000000001},"min":{"1min":11.391,"5min":9.9220000000000006,"15min":9.9220000000000006},"max":{"1min":470.80099999999999,"5min":470.80099999999999,"15min":470.80099999999999},"last":15.529999999999999},{"interface":"front","average":{"1min":49.429000000000002,"5min":22.123999999999999,"15min":17.120000000000001},"min":{"1min":6.7489999999999997,"5min":6.7489999999999997,"15min":6.7489999999999997},"max":{"1min":422.88,"5min":422.88,"15min":422.88},"last":9.0169999999999995}]},{"osd":5,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":58.246000000000002,"5min":23.637,"15min":76.748000000000005},"min":{"1min":8.8409999999999993,"5min":8.2230000000000008,"15min":5.9509999999999996},"max":{"1min":431.57900000000001,"5min":431.57900000000001,"15min":8813.2510000000002},"last":11.443},{"interface":"front","average":{"1min":62.423000000000002,"5min":22.936,"15min":77.950000000000003},"min":{"1min":6.5590000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"max":{"1min":537.68399999999997,"5min":537.68399999999997,"15min":8811.7579999999998},"last":13.138}]},{"osd":6,"last update":"Fri Oct 6 14:26:13 2023","interfaces":[{"interface":"back","average":{"1min":96.150999999999996,"5min":31.527000000000001,"15min":920.85000000000002},"min":{"1min":9.7840000000000007,"5min":6.9470000000000001,"15min":6.9470000000000001},"max":{"1min":563.85900000000004,"5min":563.85900000000004,"15min":44817.595000000001},"last":14.313000000000001},{"interface":"front","average":{"1min":84.043000000000006,"5min":30.527999999999999,"15min":922.12},"min":{"1min":12.715,"5min":7.5830000000000002,"15min":7.3789999999999996},"max":{"1min":471.80700000000002,"5min":471.80700000000002,"15min":44816.752999999997},"last":14.970000000000001}]},{"osd":7,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":50.648000000000003,"5min":23.149999999999999,"15min":1630.123},"min":{"1min":9.8119999999999994,"5min":9.7119999999999997,"15min":7.569},"max":{"1min":564.822,"5min":564.822,"15min":65382.432000000001},"last":18.446000000000002},{"interface":"front","average":{"1min":45.387999999999998,"5min":21.542999999999999,"15min":1627.549},"min":{"1min":13.016999999999999,"5min":7.8639999999999999,"15min":7.4059999999999997},"max":{"1min":433.30799999999999,"5min":433.30799999999999,"15min":65371.873},"last":20.347999999999999}]}]},{"osd":3,"up_from":834,"seq":3582002725217,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58716,"kb_used_data":5524,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313124,"statfs":{"total":96636764160,"available":96576638976,"internally_reserved":0,"allocated":5656576,"data_stored":3006177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":82,"apply_latency_ms":82,"commit_latency_ns":82000000,"apply_latency_ns":82000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":9.8170000000000002,"5min":26.867000000000001,"15min":32.284999999999997},"min":{"1min":5.6029999999999998,"5min":4.8559999999999999,"15min":4.8559999999999999},"max":{"1min":18.390999999999998,"5min":188.56700000000001,"15min":188.56700000000001},"last":22.681000000000001},{"interface":"front","average":{"1min":16.754999999999999,"5min":27.206,"15min":31.43},"min":{"1min":5.9870000000000001,"5min":5.827,"15min":5.827},"max":{"1min":135.715,"5min":214.209,"15min":214.209},"last":38.872999999999998}]},{"osd":1,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":63.271000000000001,"5min":37.026000000000003,"15min":34.704000000000001},"min":{"1min":7.7329999999999997,"5min":7.4409999999999998,"15min":7.4409999999999998},"max":{"1min":993.702,"5min":993.702,"15min":993.702},"last":122.386},{"interface":"front","average":{"1min":65.965999999999994,"5min":38.597999999999999,"15min":36.668999999999997},"min":{"1min":7.8520000000000003,"5min":6.6100000000000003,"15min":6.6100000000000003},"max":{"1min":991.50599999999997,"5min":991.50599999999997,"15min":991.50599999999997},"last":147.75800000000001}]},{"osd":2,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":21.853000000000002,"5min":29.605,"15min":34.101999999999997},"min":{"1min":8.9570000000000007,"5min":8.4199999999999999,"15min":8.4199999999999999},"max":{"1min":154.76599999999999,"5min":230.25399999999999,"15min":230.25399999999999},"last":52.75},{"interface":"front","average":{"1min":19.876000000000001,"5min":27.256,"15min":30.832000000000001},"min":{"1min":8.1959999999999997,"5min":8.1959999999999997,"15min":8.1959999999999997},"max":{"1min":95.603999999999999,"5min":165.553,"15min":165.553},"last":34.790999999999997}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":2736.415,"5min":573.53200000000004,"15min":216.17099999999999},"min":{"1min":10.223000000000001,"5min":7.1020000000000003,"15min":7.1020000000000003},"max":{"1min":40828.069000000003,"5min":40828.069000000003,"15min":40828.069000000003},"last":126.91200000000001},{"interface":"front","average":{"1min":18.699999999999999,"5min":28.707999999999998,"15min":32.601999999999997},"min":{"1min":8.6150000000000002,"5min":8.6150000000000002,"15min":8.6150000000000002},"max":{"1min":100.607,"5min":194.14599999999999,"15min":194.14599999999999},"last":83.572999999999993}]},{"osd":5,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":19.344000000000001,"5min":30.407,"15min":34.871000000000002},"min":{"1min":10.394,"5min":10.394,"15min":10.394},"max":{"1min":98.769000000000005,"5min":196.50200000000001,"15min":196.50200000000001},"last":36.985999999999997},{"interface":"front","average":{"1min":18.803000000000001,"5min":27.030999999999999,"15min":30.731000000000002},"min":{"1min":7.9329999999999998,"5min":7.9329999999999998,"15min":7.9329999999999998},"max":{"1min":111.729,"5min":177.739,"15min":177.739},"last":42.231999999999999}]},{"osd":6,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":20.492999999999999,"5min":38.835999999999999,"15min":47.399000000000001},"min":{"1min":9.6929999999999996,"5min":9.6929999999999996,"15min":9.6929999999999996},"max":{"1min":113.20399999999999,"5min":381.13,"15min":381.13},"last":127.839},{"interface":"front","average":{"1min":22.385999999999999,"5min":32.366999999999997,"15min":37.090000000000003},"min":{"1min":10.928000000000001,"5min":10.314,"15min":10.314},"max":{"1min":108.327,"5min":245.16200000000001,"15min":245.16200000000001},"last":109.922}]},{"osd":7,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":23.408999999999999,"5min":27.462,"15min":29.341999999999999},"min":{"1min":11.864000000000001,"5min":11.411,"15min":11.411},"max":{"1min":113.56699999999999,"5min":227.42599999999999,"15min":227.42599999999999},"last":81.742999999999995},{"interface":"front","average":{"1min":21.158000000000001,"5min":24.553999999999998,"15min":25.303999999999998},"min":{"1min":11.800000000000001,"5min":11.800000000000001,"15min":11.800000000000001},"max":{"1min":101.83499999999999,"5min":164.09899999999999,"15min":164.09899999999999},"last":52.259999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509831,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58660,"kb_used_data":5468,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313180,"statfs":{"total":96636764160,"available":96576696320,"internally_reserved":0,"allocated":5599232,"data_stored":2976840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":12.117000000000001,"5min":11.677,"15min":16.064},"min":{"1min":7.3819999999999997,"5min":5.452,"15min":5.3630000000000004},"max":{"1min":24.227,"5min":31.484000000000002,"15min":182.96600000000001},"last":37.688000000000002},{"interface":"front","average":{"1min":10.268000000000001,"5min":12.111000000000001,"15min":17.263000000000002},"min":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"max":{"1min":17.821000000000002,"5min":44.423000000000002,"15min":226.68600000000001},"last":27.803000000000001}]},{"osd":1,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":18.187000000000001,"5min":16.309999999999999,"15min":17.614000000000001},"min":{"1min":7.6710000000000003,"5min":7.3630000000000004,"15min":6.0970000000000004},"max":{"1min":100.065,"5min":101.874,"15min":148.20400000000001},"last":38.289000000000001},{"interface":"front","average":{"1min":12.387,"5min":14.256,"15min":17.227},"min":{"1min":6.6840000000000002,"5min":6.4630000000000001,"15min":6.3140000000000001},"max":{"1min":53.959000000000003,"5min":161.97499999999999,"15min":229.82499999999999},"last":51.273000000000003}]},{"osd":3,"last update":"Fri Oct 6 14:25:48 2023","interfaces":[{"interface":"back","average":{"1min":16.808,"5min":16.744,"15min":16.396000000000001},"min":{"1min":8.2769999999999992,"5min":8.2769999999999992,"15min":8.2769999999999992},"max":{"1min":31.460000000000001,"5min":33.399999999999999,"15min":33.399999999999999},"last":62.646999999999998},{"interface":"front","average":{"1min":13.861000000000001,"5min":14.6,"15min":14.781000000000001},"min":{"1min":8.6799999999999997,"5min":8.2509999999999994,"15min":8.2509999999999994},"max":{"1min":28.713000000000001,"5min":29.898,"15min":29.898},"last":64.054000000000002}]},{"osd":4,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":14883.07,"5min":2990.154,"15min":1010.401},"min":{"1min":6.8150000000000004,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":38405.120000000003,"5min":38405.120000000003,"15min":38405.120000000003},"last":70.638000000000005},{"interface":"front","average":{"1min":14785.589,"5min":2970.7600000000002,"15min":1006.212},"min":{"1min":7.8520000000000003,"5min":7.6550000000000002,"15min":7.6550000000000002},"max":{"1min":38264.116000000002,"5min":38264.116000000002,"15min":38264.116000000002},"last":28.445}]},{"osd":5,"last update":"Fri Oct 6 14:25:51 2023","interfaces":[{"interface":"back","average":{"1min":16.574999999999999,"5min":16.280999999999999,"15min":92.066999999999993},"min":{"1min":8.2560000000000002,"5min":8.2560000000000002,"15min":7.6589999999999998},"max":{"1min":32.328000000000003,"5min":50.210000000000001,"15min":9721.0490000000009},"last":73.159000000000006},{"interface":"front","average":{"1min":14.137,"5min":16.710999999999999,"15min":96.823999999999998},"min":{"1min":8.6099999999999994,"5min":8.6099999999999994,"15min":8.6099999999999994},"max":{"1min":24.001000000000001,"5min":73.320999999999998,"15min":9793.7890000000007},"last":77.673000000000002}]},{"osd":6,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":14.999000000000001,"5min":17.789000000000001,"15min":1309.6320000000001},"min":{"1min":8.8689999999999998,"5min":8.8689999999999998,"15min":8.8689999999999998},"max":{"1min":23.431999999999999,"5min":44.606999999999999,"15min":44531.741000000002},"last":34.725999999999999},{"interface":"front","average":{"1min":16.434000000000001,"5min":16.972000000000001,"15min":1305.6900000000001},"min":{"1min":7.5330000000000004,"5min":7.141,"15min":7.141},"max":{"1min":49.576999999999998,"5min":49.576999999999998,"15min":44501.423000000003},"last":76.936000000000007}]},{"osd":7,"last update":"Fri Oct 6 14:25:34 2023","interfaces":[{"interface":"back","average":{"1min":20.236000000000001,"5min":20.585000000000001,"15min":1733.5730000000001},"min":{"1min":12.177,"5min":10.631,"15min":8.6189999999999998},"max":{"1min":38.399999999999999,"5min":61.658999999999999,"15min":69773.953999999998},"last":73.903000000000006},{"interface":"front","average":{"1min":21.065999999999999,"5min":18.300999999999998,"15min":1723.3030000000001},"min":{"1min":11.403,"5min":9.5909999999999993,"15min":8.5429999999999993},"max":{"1min":31.567,"5min":34.923000000000002,"15min":69547.547999999995},"last":78.433999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62840,"kb_used_data":5552,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94309000,"statfs":{"total":96636764160,"available":96572416000,"internally_reserved":0,"allocated":5685248,"data_stored":3054367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":56,"apply_latency_ms":56,"commit_latency_ns":56000000,"apply_latency_ns":56000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":8.6549999999999994,"5min":11.162000000000001,"15min":18.341000000000001},"min":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.4050000000000002},"max":{"1min":12.99,"5min":94.040999999999997,"15min":574.78800000000001},"last":5.0999999999999996},{"interface":"front","average":{"1min":11.332000000000001,"5min":15.573,"15min":21.341999999999999},"min":{"1min":7.5309999999999997,"5min":7.5309999999999997,"15min":4.5140000000000002},"max":{"1min":17.783999999999999,"5min":130.036,"15min":522.72699999999998},"last":7.6779999999999999}]},{"osd":1,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.231,"5min":27.030999999999999,"15min":24.16},"min":{"1min":7.3150000000000004,"5min":5.4059999999999997,"15min":5.4059999999999997},"max":{"1min":14.238,"5min":1271.6179999999999,"15min":1271.6179999999999},"last":9.1989999999999998},{"interface":"front","average":{"1min":12.869,"5min":30.803000000000001,"15min":27.870999999999999},"min":{"1min":9.9900000000000002,"5min":6.8639999999999999,"15min":6.8639999999999999},"max":{"1min":17.446999999999999,"5min":1322.71,"15min":1322.71},"last":12.44}]},{"osd":2,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.842000000000001,"5min":14.388999999999999,"15min":19.920999999999999},"min":{"1min":9.4809999999999999,"5min":6.1660000000000004,"15min":5.532},"max":{"1min":30.937999999999999,"5min":101.197,"15min":525.81600000000003},"last":16.143999999999998},{"interface":"front","average":{"1min":11.125999999999999,"5min":17.469999999999999,"15min":24.055},"min":{"1min":6.7409999999999997,"5min":6.4509999999999996,"15min":6.4509999999999996},"max":{"1min":27.02,"5min":99.492999999999995,"15min":468.70400000000001},"last":9.9740000000000002}]},{"osd":3,"last update":"Fri Oct 6 14:25:41 2023","interfaces":[{"interface":"back","average":{"1min":15.800000000000001,"5min":20.021000000000001,"15min":20.113},"min":{"1min":6.8470000000000004,"5min":6.8470000000000004,"15min":6.8470000000000004},"max":{"1min":40.512,"5min":126.55500000000001,"15min":126.55500000000001},"last":17.16},{"interface":"front","average":{"1min":14.677,"5min":21.684999999999999,"15min":23.826000000000001},"min":{"1min":6.0389999999999997,"5min":6.0389999999999997,"15min":6.0389999999999997},"max":{"1min":49.473999999999997,"5min":236.27000000000001,"15min":236.27000000000001},"last":11.718}]},{"osd":4,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15821.066000000001,"5min":3179.7730000000001,"15min":1076.7719999999999},"min":{"1min":10.141,"5min":9.1189999999999998,"15min":8.7490000000000006},"max":{"1min":38720.860999999997,"5min":38720.860999999997,"15min":38720.860999999997},"last":14.757},{"interface":"front","average":{"1min":15816.687,"5min":3177.5610000000001,"15min":1074.1969999999999},"min":{"1min":6.782,"5min":6.782,"15min":6.782},"max":{"1min":38719.173000000003,"5min":38719.173000000003,"15min":38719.173000000003},"last":13.971}]},{"osd":5,"last update":"Fri Oct 6 14:25:51 2023","interfaces":[{"interface":"back","average":{"1min":17.318999999999999,"5min":21.241,"15min":107.18600000000001},"min":{"1min":8.8800000000000008,"5min":8.8800000000000008,"15min":8.8800000000000008},"max":{"1min":43.317999999999998,"5min":219.06200000000001,"15min":8032.4189999999999},"last":11.210000000000001},{"interface":"front","average":{"1min":18.963999999999999,"5min":21.207999999999998,"15min":109.759},"min":{"1min":7.4409999999999998,"5min":7.1459999999999999,"15min":7.1459999999999999},"max":{"1min":69.081000000000003,"5min":171.041,"15min":8030.5749999999998},"last":16.884}]},{"osd":7,"last update":"Fri Oct 6 14:25:37 2023","interfaces":[{"interface":"back","average":{"1min":20.959,"5min":19.791,"15min":1130.614},"min":{"1min":11.01,"5min":11.01,"15min":11.01},"max":{"1min":45.648000000000003,"5min":93.355999999999995,"15min":69750.712},"last":18.297000000000001},{"interface":"front","average":{"1min":16.279,"5min":14.369999999999999,"15min":1124.646},"min":{"1min":7.1769999999999996,"5min":6.1070000000000002,"15min":5.984},"max":{"1min":40.57,"5min":121.146,"15min":69749.794999999998},"last":10.629}]}]},{"osd":5,"up_from":659,"seq":2830383448439,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77156,"kb_used_data":5916,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94294684,"statfs":{"total":96636764160,"available":96557756416,"internally_reserved":0,"allocated":6057984,"data_stored":3418788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":65,"apply_latency_ms":65,"commit_latency_ns":65000000,"apply_latency_ns":65000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":10.645,"5min":13.339,"15min":38.990000000000002},"min":{"1min":4.6189999999999998,"5min":4.3339999999999996,"15min":4.1879999999999997},"max":{"1min":22.547999999999998,"5min":173.54599999999999,"15min":680.28200000000004},"last":7.141},{"interface":"front","average":{"1min":7.4779999999999998,"5min":11.548999999999999,"15min":29.652000000000001},"min":{"1min":4.7430000000000003,"5min":4.5579999999999998,"15min":4.5579999999999998},"max":{"1min":16.622,"5min":154.36199999999999,"15min":608.46199999999999},"last":14.422000000000001}]},{"osd":1,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":45.543999999999997,"5min":20.469000000000001,"15min":43.674999999999997},"min":{"1min":6.3620000000000001,"5min":5.4930000000000003,"15min":5.4930000000000003},"max":{"1min":633.78700000000003,"5min":633.78700000000003,"15min":633.78700000000003},"last":12.051},{"interface":"front","average":{"1min":46.32,"5min":23.73,"15min":46.381},"min":{"1min":10.032,"5min":5.9000000000000004,"15min":5.8259999999999996},"max":{"1min":598.60000000000002,"5min":598.60000000000002,"15min":701.19399999999996},"last":11.423}]},{"osd":2,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":14.177,"5min":14.871,"15min":44.731999999999999},"min":{"1min":8.218,"5min":6.8369999999999997,"15min":6.6029999999999998},"max":{"1min":27.632000000000001,"5min":87.468000000000004,"15min":603.024},"last":16.187999999999999},{"interface":"front","average":{"1min":15.343,"5min":17.352,"15min":49.799999999999997},"min":{"1min":10.210000000000001,"5min":8.1289999999999996,"15min":8.1289999999999996},"max":{"1min":29.491,"5min":84.820999999999998,"15min":690.71799999999996},"last":19.492000000000001}]},{"osd":3,"last update":"Fri Oct 6 14:25:42 2023","interfaces":[{"interface":"back","average":{"1min":15.289999999999999,"5min":28.317,"15min":32.777999999999999},"min":{"1min":9.0920000000000005,"5min":8.1690000000000005,"15min":8.1690000000000005},"max":{"1min":31.006,"5min":264.37900000000002,"15min":264.37900000000002},"last":20.670000000000002},{"interface":"front","average":{"1min":16.370000000000001,"5min":27.132999999999999,"15min":30.166},"min":{"1min":11.48,"5min":9.0990000000000002,"15min":9.0990000000000002},"max":{"1min":33.726999999999997,"5min":173.858,"15min":173.858},"last":18.087}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":3491.913,"5min":712.81600000000003,"15min":274.36799999999999},"min":{"1min":10.141,"5min":7.9340000000000002,"15min":7.9340000000000002},"max":{"1min":38256.345000000001,"5min":38256.345000000001,"15min":38256.345000000001},"last":22.091999999999999},{"interface":"front","average":{"1min":3489.7570000000001,"5min":714.00800000000004,"15min":281.79899999999998},"min":{"1min":13.077999999999999,"5min":9.1750000000000007,"15min":8.1839999999999993},"max":{"1min":38222.745999999999,"5min":38222.745999999999,"15min":38222.745999999999},"last":12.853}]},{"osd":6,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":15.797000000000001,"5min":18.550999999999998,"15min":47.345999999999997},"min":{"1min":10.766,"5min":7.7300000000000004,"15min":7.7300000000000004},"max":{"1min":27.385000000000002,"5min":108.053,"15min":607.87400000000002},"last":16.995999999999999},{"interface":"front","average":{"1min":17.838000000000001,"5min":21.890000000000001,"15min":51.551000000000002},"min":{"1min":14.509,"5min":10.855,"15min":7.1040000000000001},"max":{"1min":26.152000000000001,"5min":91.733000000000004,"15min":739.33100000000002},"last":20.09}]},{"osd":7,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":17.960999999999999,"5min":21.77,"15min":47.536000000000001},"min":{"1min":10.858000000000001,"5min":9.3569999999999993,"15min":7.0350000000000001},"max":{"1min":32.581000000000003,"5min":90.037000000000006,"15min":609.59799999999996},"last":15.159000000000001},{"interface":"front","average":{"1min":9.0069999999999997,"5min":13.292,"15min":44.540999999999997},"min":{"1min":6.4420000000000002,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":13.343999999999999,"5min":77.034999999999997,"15min":610.43799999999999},"last":10.16}]}]},{"osd":7,"up_from":13,"seq":55834575233,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80860,"kb_used_data":5524,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94290980,"statfs":{"total":96636764160,"available":96553963520,"internally_reserved":0,"allocated":5656576,"data_stored":3006177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:25:26 2023","interfaces":[{"interface":"back","average":{"1min":8.4930000000000003,"5min":10.589,"15min":33.819000000000003},"min":{"1min":4.907,"5min":4.742,"15min":4.2539999999999996},"max":{"1min":14.169,"5min":46.529000000000003,"15min":930.62099999999998},"last":8.2639999999999993},{"interface":"front","average":{"1min":14.048,"5min":11.707000000000001,"15min":31.795999999999999},"min":{"1min":6.6950000000000003,"5min":4.8079999999999998,"15min":4.8079999999999998},"max":{"1min":69.477999999999994,"5min":69.477999999999994,"15min":879.35199999999998},"last":7.601}]},{"osd":1,"last update":"Fri Oct 6 14:25:26 2023","interfaces":[{"interface":"back","average":{"1min":70.927000000000007,"5min":26.693999999999999,"15min":40.520000000000003},"min":{"1min":5.4790000000000001,"5min":5.4790000000000001,"15min":5.4790000000000001},"max":{"1min":1094.5889999999999,"5min":1094.5889999999999,"15min":1094.5889999999999},"last":11.683},{"interface":"front","average":{"1min":69.433000000000007,"5min":25.927,"15min":25.18},"min":{"1min":8.3170000000000002,"5min":6.2270000000000003,"15min":5.4539999999999997},"max":{"1min":1057.856,"5min":1057.856,"15min":1057.856},"last":14.989000000000001}]},{"osd":2,"last update":"Fri Oct 6 14:25:26 2023","interfaces":[{"interface":"back","average":{"1min":16.998000000000001,"5min":15.606999999999999,"15min":24.332000000000001},"min":{"1min":9.0139999999999993,"5min":6.5149999999999997,"15min":6.202},"max":{"1min":75.614000000000004,"5min":75.823999999999998,"15min":316.84399999999999},"last":6.9969999999999999},{"interface":"front","average":{"1min":18.135999999999999,"5min":16.129999999999999,"15min":40.203000000000003},"min":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"max":{"1min":78.471999999999994,"5min":98.183000000000007,"15min":931.94500000000005},"last":9.9499999999999993}]},{"osd":3,"last update":"Fri Oct 6 14:25:47 2023","interfaces":[{"interface":"back","average":{"1min":14.034000000000001,"5min":20.844000000000001,"15min":22.948},"min":{"1min":6.9969999999999999,"5min":6.9969999999999999,"15min":6.9969999999999999},"max":{"1min":40.936,"5min":184.93700000000001,"15min":184.93700000000001},"last":10.542999999999999},{"interface":"front","average":{"1min":13.778,"5min":18.068999999999999,"15min":18.817},"min":{"1min":8.2010000000000005,"5min":7.6699999999999999,"15min":7.6699999999999999},"max":{"1min":32.956000000000003,"5min":77.144000000000005,"15min":77.144000000000005},"last":15.557}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":2132.1480000000001,"5min":439.87099999999998,"15min":167.66999999999999},"min":{"1min":7.8869999999999996,"5min":5.9429999999999996,"15min":5.9429999999999996},"max":{"1min":40210.642,"5min":40210.642,"15min":40210.642},"last":9.2669999999999995},{"interface":"front","average":{"1min":2144.9340000000002,"5min":442.64400000000001,"15min":163.51599999999999},"min":{"1min":11.497999999999999,"5min":7.4180000000000001,"15min":6.9279999999999999},"max":{"1min":40377.220000000001,"5min":40377.220000000001,"15min":40377.220000000001},"last":17.106999999999999}]},{"osd":5,"last update":"Fri Oct 6 14:25:50 2023","interfaces":[{"interface":"back","average":{"1min":17.524000000000001,"5min":18.869,"15min":51.777000000000001},"min":{"1min":8.4939999999999998,"5min":8.1229999999999993,"15min":7.6909999999999998},"max":{"1min":70.796000000000006,"5min":149.61799999999999,"15min":6750.3770000000004},"last":11.081},{"interface":"front","average":{"1min":19.053999999999998,"5min":19.542000000000002,"15min":52.655000000000001},"min":{"1min":7.9029999999999996,"5min":6.9290000000000003,"15min":6.9290000000000003},"max":{"1min":83.019000000000005,"5min":174.839,"15min":6875.5780000000004},"last":16.416}]},{"osd":6,"last update":"Fri Oct 6 14:25:26 2023","interfaces":[{"interface":"back","average":{"1min":20.823,"5min":17.948,"15min":27.071999999999999},"min":{"1min":11.436999999999999,"5min":7.9509999999999996,"15min":7.7290000000000001},"max":{"1min":75.013999999999996,"5min":75.013999999999996,"15min":380.74000000000001},"last":14.266},{"interface":"front","average":{"1min":23.314,"5min":19.940999999999999,"15min":44.384999999999998},"min":{"1min":9.1120000000000001,"5min":8.9489999999999998,"15min":8.9489999999999998},"max":{"1min":83.512,"5min":120.83799999999999,"15min":935.49300000000005},"last":12.444000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T14:26:26.292 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T14:26:26.292 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T14:26:26.399 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.397+0000 7fd49d4b4640 10 monclient: get_monmap_and_config 2023-10-06T14:26:26.399 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.397+0000 7fd49d4b4640 10 monclient: build_initial_monmap 2023-10-06T14:26:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.397+0000 7fd49d4b4640 10 monclient: monmap: 2023-10-06T14:26:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:26.398731+0000 2023-10-06T14:26:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:26.398731+0000 2023-10-06T14:26:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:26.401 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 1 Processor -- start 2023-10-06T14:26:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 1 -- start start 2023-10-06T14:26:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 10 monclient: init 2023-10-06T14:26:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 10 monclient: _add_conns ranks=[2,3,0,4,1] 2023-10-06T14:26:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd49806b6d0 0x7fd498144200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 10 monclient(hunting): picked mon.noname-b con 0x7fd49806b6d0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd498144740 0x7fd49813df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 10 monclient(hunting): picked mon.noname-a con 0x7fd498144740 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd49813e460 addr v1:172.21.15.19:6789/0 2023-10-06T14:26:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49806baa0 con 0x7fd49813e460 2023-10-06T14:26:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4980b2530 con 0x7fd498144740 2023-10-06T14:26:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980b2810 con 0x7fd49806b6d0 2023-10-06T14:26:26.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:26.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.398+0000 7fd49d4b4640 10 monclient(hunting): authenticate will time out at 3314.585309s 2023-10-06T14:26:26.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.399+0000 7fd496ffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd49806b6d0 0x7fd498144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:26.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.399+0000 7fd48e7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd498144740 0x7fd49813df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:26.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.399+0000 7fd496ffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd49806b6d0 0x7fd498144200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40212/0 (socket says 172.21.15.19:40212) 2023-10-06T14:26:26.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.399+0000 7fd496ffd640 1 -- 172.21.15.19:0/1952577257 learned_addr learned my addr 172.21.15.19:0/1952577257 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:26.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.399+0000 7fd496ffd640 10 monclient(hunting): get_auth_request con 0x7fd49806b6d0 auth_method 0 2023-10-06T14:26:26.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.399+0000 7fd496ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:26.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.399+0000 7fd496ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:26.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.399+0000 7fd496ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.399+0000 7fd48e7fc640 10 monclient(hunting): get_auth_request con 0x7fd498144740 auth_method 0 2023-10-06T14:26:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.399+0000 7fd48e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.399+0000 7fd48e7fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.399+0000 7fd48e7fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.400+0000 7fd496ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.400+0000 7fd496ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.400+0000 7fd496ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.400+0000 7fd48e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.400+0000 7fd48e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.400+0000 7fd48e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:26.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.401+0000 7fd496ffd640 10 monclient(hunting): handle_auth_done global_id 17120 payload 274 2023-10-06T14:26:26.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.401+0000 7fd496ffd640 1 -- 172.21.15.19:0/1952577257 >> v2:172.21.15.19:3300/0 conn(0x7fd498144740 msgr2=0x7fd49813df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:26.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.401+0000 7fd496ffd640 1 --2- 172.21.15.19:0/1952577257 >> v2:172.21.15.19:3300/0 conn(0x7fd498144740 0x7fd49813df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.402+0000 7fd496ffd640 1 -- 172.21.15.19:0/1952577257 >> v1:172.21.15.19:6789/0 conn(0x7fd49813e460 legacy=0x7fd4980b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:26.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.402+0000 7fd496ffd640 10 monclient: _finish_hunting 0 2023-10-06T14:26:26.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.402+0000 7fd496ffd640 1 monclient: found mon.noname-b 2023-10-06T14:26:26.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.402+0000 7fd496ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:26.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.402+0000 7fd496ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T14:26:26.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.402+0000 7fd496ffd640 1 -- 172.21.15.19:0/1952577257 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4980aeff0 con 0x7fd49806b6d0 2023-10-06T14:26:26.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.404+0000 7fd496ffd640 1 --2- 172.21.15.19:0/1952577257 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd49806b6d0 0x7fd498144200 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 crypto rx=0x7fd488009870 tx=0x7fd48802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0d59631299be3dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:26.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.407+0000 7fd4967fc640 1 -- 172.21.15.19:0/1952577257 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488031640 con 0x7fd49806b6d0 2023-10-06T14:26:26.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.407+0000 7fd4967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:26.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd4967fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:26.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd4967fc640 10 monclient: dump: 2023-10-06T14:26:26.410 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:26.410 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:26.410 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:26.410 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:26.410 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:26.411 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd4967fc640 10 monclient: _finish_auth 0 2023-10-06T14:26:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd4967fc640 10 monclient: _check_auth_tickets 2023-10-06T14:26:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd4967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd4967fc640 1 -- 172.21.15.19:0/1952577257 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4880317a0 con 0x7fd49806b6d0 2023-10-06T14:26:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd4967fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd4967fc640 1 -- 172.21.15.19:0/1952577257 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488031b30 con 0x7fd49806b6d0 2023-10-06T14:26:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd49d4b4640 5 monclient: authenticate success, global_id 17120 2023-10-06T14:26:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd49d4b4640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd49d4b4640 4 set_mon_vals no callback set 2023-10-06T14:26:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd4967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:26.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd4967fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:26.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd4967fc640 10 monclient: dump: 2023-10-06T14:26:26.414 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:26.414 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:26.414 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:26.414 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:26.415 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:26.415 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:26.415 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:26.415 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:26.415 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:26.415 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:26.420 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd49d4b4640 10 monclient: shutdown 2023-10-06T14:26:26.420 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd49d4b4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:26.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd49d4b4640 1 -- 172.21.15.19:0/1952577257 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd49806b6d0 msgr2=0x7fd498144200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:26.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd49d4b4640 1 --2- 172.21.15.19:0/1952577257 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd49806b6d0 0x7fd498144200 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 crypto rx=0x7fd488009870 tx=0x7fd48802b8f0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd49d4b4640 1 -- 172.21.15.19:0/1952577257 shutdown_connections 2023-10-06T14:26:26.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd49d4b4640 1 --2- 172.21.15.19:0/1952577257 >> v2:172.21.15.19:3300/0 conn(0x7fd498144740 0x7fd49813df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd49d4b4640 1 --2- 172.21.15.19:0/1952577257 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd49806b6d0 0x7fd498144200 unknown :-1 s=CLOSED pgs=3117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd49d4b4640 1 -- 172.21.15.19:0/1952577257 >> 172.21.15.19:0/1952577257 conn(0x7fd4980665b0 msgr2=0x7fd49806ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:26.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd49d4b4640 1 -- 172.21.15.19:0/1952577257 shutdown_connections 2023-10-06T14:26:26.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.408+0000 7fd49d4b4640 1 -- 172.21.15.19:0/1952577257 wait complete. 2023-10-06T14:26:26.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.409+0000 7fd49d4b4640 10 monclient: build_initial_monmap 2023-10-06T14:26:26.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.409+0000 7fd49d4b4640 10 monclient: monmap: 2023-10-06T14:26:26.422 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:26.422 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:26.422 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:26.422 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:26.422 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:26.422 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:26.422 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:26.422 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:26.422 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:26.423 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:26.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.409+0000 7fd49d4b4640 1 librados: starting msgr at 2023-10-06T14:26:26.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.409+0000 7fd49d4b4640 1 librados: starting objecter 2023-10-06T14:26:26.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.409+0000 7fd49d4b4640 1 Processor -- start 2023-10-06T14:26:26.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 1 -- start start 2023-10-06T14:26:26.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 1 librados: setting wanted keys 2023-10-06T14:26:26.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 1 librados: calling monclient init 2023-10-06T14:26:26.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 10 monclient: init 2023-10-06T14:26:26.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:26.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 10 monclient: _add_conns ranks=[2,0,1] 2023-10-06T14:26:26.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd49806b6d0 0x7fd4980ac960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:26.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 10 monclient(hunting): picked mon.noname-c con 0x7fd49806b6d0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:26.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd49813e460 0x7fd4981cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:26.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 10 monclient(hunting): picked mon.noname-a con 0x7fd49813e460 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:26.425 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd498144740 0x7fd4981d69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:26.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 10 monclient(hunting): picked mon.noname-b con 0x7fd498144740 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:26.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:26.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4981d6ee0 con 0x7fd49813e460 2023-10-06T14:26:26.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:26.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4981d6480 con 0x7fd498144740 2023-10-06T14:26:26.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:26.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4981cc7c0 con 0x7fd49806b6d0 2023-10-06T14:26:26.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:26.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.410+0000 7fd49d4b4640 10 monclient(hunting): authenticate will time out at 3314.597074s 2023-10-06T14:26:26.426 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd496ffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd49806b6d0 0x7fd4980ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd48e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd49813e460 0x7fd4981cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd496ffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd49806b6d0 0x7fd4980ac960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40218/0 (socket says 172.21.15.19:40218) 2023-10-06T14:26:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd496ffd640 1 -- 172.21.15.19:0/3882016261 learned_addr learned my addr 172.21.15.19:0/3882016261 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd496ffd640 10 monclient(hunting): get_auth_request con 0x7fd49806b6d0 auth_method 0 2023-10-06T14:26:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd496ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd496ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd496ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd4977fe640 1 --2- 172.21.15.19:0/3882016261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd498144740 0x7fd4981d69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:26.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd48e7fc640 10 monclient(hunting): get_auth_request con 0x7fd49813e460 auth_method 0 2023-10-06T14:26:26.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd48e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:26.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd48e7fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:26.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd48e7fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:26.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd4977fe640 10 monclient(hunting): get_auth_request con 0x7fd498144740 auth_method 0 2023-10-06T14:26:26.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd4977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:26.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd4977fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:26.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd4977fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:26.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd496ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:26.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.411+0000 7fd496ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:26.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.412+0000 7fd496ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:26.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.412+0000 7fd4977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:26.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.412+0000 7fd4977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:26.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.412+0000 7fd4977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:26.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.412+0000 7fd48e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:26.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.412+0000 7fd48e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:26.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.412+0000 7fd48e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:26.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.414+0000 7fd496ffd640 10 monclient(hunting): handle_auth_done global_id 17123 payload 824 2023-10-06T14:26:26.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.414+0000 7fd496ffd640 1 -- 172.21.15.19:0/3882016261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd498144740 msgr2=0x7fd4981d69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:26.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.414+0000 7fd496ffd640 1 --2- 172.21.15.19:0/3882016261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd498144740 0x7fd4981d69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.414+0000 7fd496ffd640 1 -- 172.21.15.19:0/3882016261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd49813e460 msgr2=0x7fd4981cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:26.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.414+0000 7fd496ffd640 1 --2- 172.21.15.19:0/3882016261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd49813e460 0x7fd4981cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.414+0000 7fd496ffd640 10 monclient: _finish_hunting 0 2023-10-06T14:26:26.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.414+0000 7fd496ffd640 1 monclient: found mon.noname-c 2023-10-06T14:26:26.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.414+0000 7fd496ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:26.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.414+0000 7fd496ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T14:26:26.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.414+0000 7fd496ffd640 1 -- 172.21.15.19:0/3882016261 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4880094d0 con 0x7fd49806b6d0 2023-10-06T14:26:26.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.417+0000 7fd496ffd640 1 --2- 172.21.15.19:0/3882016261 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd49806b6d0 0x7fd4980ac960 secure :-1 s=READY pgs=3118 cs=0 l=1 rev1=1 crypto rx=0x7fd4880090b0 tx=0x7fd4880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:26.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.417+0000 7fd48ffff640 10 client.?.objecter ms_handle_connect 0x7fd49806b6d0 2023-10-06T14:26:26.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.417+0000 7fd48ffff640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd48ffff640 1 -- 172.21.15.19:0/3882016261 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48803a9f0 con 0x7fd49806b6d0 2023-10-06T14:26:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd48ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd48ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T14:26:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd48ffff640 10 monclient: dump: 2023-10-06T14:26:26.433 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:26.433 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:26.433 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:26.433 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:26.433 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:26.433 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:26.433 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:26.433 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:26.433 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:26.434 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd48ffff640 10 monclient: _finish_auth 0 2023-10-06T14:26:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd48ffff640 10 monclient: _check_auth_tickets 2023-10-06T14:26:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd48ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd48ffff640 1 -- 172.21.15.19:0/3882016261 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48803ab70 con 0x7fd49806b6d0 2023-10-06T14:26:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd49d4b4640 5 monclient: authenticate success, global_id 17123 2023-10-06T14:26:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd49d4b4640 10 monclient: _renew_subs 2023-10-06T14:26:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd49d4b4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd49d4b4640 1 -- 172.21.15.19:0/3882016261 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4981d6030 con 0x7fd49806b6d0 2023-10-06T14:26:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd48ffff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd48ffff640 1 -- 172.21.15.19:0/3882016261 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4880443f0 con 0x7fd49806b6d0 2023-10-06T14:26:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd48ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd48ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd48ffff640 10 monclient: dump: 2023-10-06T14:26:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:26.436 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd495ffb640 4 set_mon_vals no callback set 2023-10-06T14:26:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd49d4b4640 10 client.17123.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd49d4b4640 10 monclient: _renew_subs 2023-10-06T14:26:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd49d4b4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd49d4b4640 1 -- 172.21.15.19:0/3882016261 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49813ed50 con 0x7fd49806b6d0 2023-10-06T14:26:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.421+0000 7fd49d4b4640 1 librados: init done 2023-10-06T14:26:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.422+0000 7fd49d4b4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.422+0000 7fd49d4b4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.422+0000 7fd49d4b4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.422+0000 7fd49d4b4640 1 -- 172.21.15.19:0/3882016261 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4980678a0 con 0x7fd49806b6d0 2023-10-06T14:26:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.440+0000 7fd48ffff640 1 -- 172.21.15.19:0/3882016261 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd48802f050 con 0x7fd49806b6d0 2023-10-06T14:26:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.440+0000 7fd48ffff640 1 --2- 172.21.15.19:0/3882016261 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd47806c1c0 0x7fd47806e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.440+0000 7fd48e7fc640 1 --2- 172.21.15.19:0/3882016261 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd47806c1c0 0x7fd47806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.440+0000 7fd48e7fc640 10 monclient: get_auth_request con 0x7fd47806c1c0 auth_method 0 2023-10-06T14:26:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.441+0000 7fd48ffff640 10 client.17123.objecter ms_handle_connect 0x7fd47806c1c0 2023-10-06T14:26:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.441+0000 7fd48e7fc640 1 --2- 172.21.15.19:0/3882016261 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd47806c1c0 0x7fd47806e670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd4980a8fa0 tx=0x7fd480009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.443+0000 7fd48ffff640 1 -- 172.21.15.19:0/3882016261 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(969..969 src has 595..969) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fd48807a140 con 0x7fd49806b6d0 2023-10-06T14:26:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.443+0000 7fd48ffff640 10 client.17123.objecter ms_dispatch 0x7fd498000b90 osd_map(969..969 src has 595..969) v4 2023-10-06T14:26:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.443+0000 7fd48ffff640 3 client.17123.objecter handle_osd_map got epochs [969,969] > 0 2023-10-06T14:26:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.443+0000 7fd48ffff640 3 client.17123.objecter handle_osd_map decoding full epoch 969 2023-10-06T14:26:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.443+0000 7fd48ffff640 20 client.17123.objecter dump_active .. 0 homeless 2023-10-06T14:26:26.700 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.698+0000 7fd48ffff640 1 -- 172.21.15.19:0/3882016261 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7fd488076480 con 0x7fd49806b6d0 2023-10-06T14:26:26.701 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.698+0000 7fd48ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:26.701 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.698+0000 7fd48ffff640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:26:26.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.872+0000 7fd49d4b4640 1 -- 172.21.15.19:0/3882016261 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4980b1040 con 0x7fd47806c1c0 2023-10-06T14:26:26.878 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.877+0000 7fd48ffff640 1 -- 172.21.15.19:0/3882016261 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+151924 (secure 0 0 0) 0x7fd4800119d0 con 0x7fd47806c1c0 2023-10-06T14:26:26.880 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T14:26:26.881 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2023-10-06T14:26:26.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 10 librados: watch_flush enter 2023-10-06T14:26:26.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 10 librados: watch_flush exit 2023-10-06T14:26:26.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 20 client.17123.objecter shutdown clearing up homeless session... 2023-10-06T14:26:26.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 10 client.17123.objecter successfully canceled tick 2023-10-06T14:26:26.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 1 -- 172.21.15.19:0/3882016261 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd47806c1c0 msgr2=0x7fd47806e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:26.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 1 --2- 172.21.15.19:0/3882016261 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd47806c1c0 0x7fd47806e670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd4980a8fa0 tx=0x7fd480009190 comp rx=0 tx=0).stop 2023-10-06T14:26:26.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 10 monclient: shutdown 2023-10-06T14:26:26.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:26.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 1 -- 172.21.15.19:0/3882016261 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd49806b6d0 msgr2=0x7fd4980ac960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:26.883 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 1 --2- 172.21.15.19:0/3882016261 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd49806b6d0 0x7fd4980ac960 secure :-1 s=READY pgs=3118 cs=0 l=1 rev1=1 crypto rx=0x7fd4880090b0 tx=0x7fd4880046c0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.884 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 1 -- 172.21.15.19:0/3882016261 shutdown_connections 2023-10-06T14:26:26.884 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 1 --2- 172.21.15.19:0/3882016261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd498144740 0x7fd4981d69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.884 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 1 --2- 172.21.15.19:0/3882016261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd49813e460 0x7fd4981cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.884 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 1 --2- 172.21.15.19:0/3882016261 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd47806c1c0 0x7fd47806e670 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.884 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 1 --2- 172.21.15.19:0/3882016261 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd49806b6d0 0x7fd4980ac960 unknown :-1 s=CLOSED pgs=3118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:26.884 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 1 -- 172.21.15.19:0/3882016261 >> 172.21.15.19:0/3882016261 conn(0x7fd4980665b0 msgr2=0x7fd49813be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:26.884 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 1 -- 172.21.15.19:0/3882016261 shutdown_connections 2023-10-06T14:26:26.884 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 1 -- 172.21.15.19:0/3882016261 wait complete. 2023-10-06T14:26:26.884 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.879+0000 7fd49d4b4640 1 librados: shutdown 2023-10-06T14:26:26.892 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":1972,"stamp":"2023-10-06T14:26:25.803802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":514760,"kb_used_data":44936,"kb_used_omap":0,"kb_used_meta":469760,"kb_avail":754459960,"statfs":{"total":773094113280,"available":772566999040,"internally_reserved":0,"allocated":46014464,"data_stored":24890348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":481034240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":406,"apply_latency_ms":406,"commit_latency_ns":406000000,"apply_latency_ns":406000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4000299,"num_objects":-44,"num_object_clones":-1,"num_object_copies":-132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44,"num_whiteouts":0,"num_read":-31,"num_read_kb":-11,"num_write":-71,"num_write_kb":-3969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.114550"},"pg_stats":[{"pgid":"8.14","version":"0'0","reported_seq":921,"reported_epoch":966,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.757496+0000","last_change":"2023-10-06T13:54:27.674098+0000","last_active":"2023-10-06T14:26:21.757496+0000","last_peered":"2023-10-06T14:26:21.757496+0000","last_clean":"2023-10-06T14:26:21.757496+0000","last_became_active":"2023-10-06T13:54:27.664679+0000","last_became_peered":"2023-10-06T13:54:27.664679+0000","last_unstale":"2023-10-06T14:26:21.757496+0000","last_undegraded":"2023-10-06T14:26:21.757496+0000","last_fullsized":"2023-10-06T14:26:21.757496+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:57:24.020996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":920,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.836623+0000","last_change":"2023-10-06T14:22:39.751625+0000","last_active":"2023-10-06T14:26:22.836623+0000","last_peered":"2023-10-06T14:26:22.836623+0000","last_clean":"2023-10-06T14:26:22.836623+0000","last_became_active":"2023-10-06T14:22:39.724017+0000","last_became_peered":"2023-10-06T14:22:39.724017+0000","last_unstale":"2023-10-06T14:26:22.836623+0000","last_undegraded":"2023-10-06T14:26:22.836623+0000","last_fullsized":"2023-10-06T14:26:22.836623+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:31.357267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":924,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.734022+0000","last_change":"2023-10-06T14:16:33.556535+0000","last_active":"2023-10-06T14:26:20.734022+0000","last_peered":"2023-10-06T14:26:20.734022+0000","last_clean":"2023-10-06T14:26:20.734022+0000","last_became_active":"2023-10-06T14:16:33.542923+0000","last_became_peered":"2023-10-06T14:16:33.542923+0000","last_unstale":"2023-10-06T14:26:20.734022+0000","last_undegraded":"2023-10-06T14:26:20.734022+0000","last_fullsized":"2023-10-06T14:26:20.734022+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:56:19.999321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":927,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.813484+0000","last_change":"2023-10-06T13:54:29.208698+0000","last_active":"2023-10-06T14:26:22.813484+0000","last_peered":"2023-10-06T14:26:22.813484+0000","last_clean":"2023-10-06T14:26:22.813484+0000","last_became_active":"2023-10-06T13:54:29.163161+0000","last_became_peered":"2023-10-06T13:54:29.163161+0000","last_unstale":"2023-10-06T14:26:22.813484+0000","last_undegraded":"2023-10-06T14:26:22.813484+0000","last_fullsized":"2023-10-06T14:26:22.813484+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:22.518227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":919,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.795205+0000","last_change":"2023-10-06T14:22:39.220927+0000","last_active":"2023-10-06T14:26:22.795205+0000","last_peered":"2023-10-06T14:26:22.795205+0000","last_clean":"2023-10-06T14:26:22.795205+0000","last_became_active":"2023-10-06T14:22:39.128927+0000","last_became_peered":"2023-10-06T14:22:39.128927+0000","last_unstale":"2023-10-06T14:26:22.795205+0000","last_undegraded":"2023-10-06T14:26:22.795205+0000","last_fullsized":"2023-10-06T14:26:22.795205+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:02.237203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":920,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.885756+0000","last_change":"2023-10-06T14:22:39.468045+0000","last_active":"2023-10-06T14:26:22.885756+0000","last_peered":"2023-10-06T14:26:22.885756+0000","last_clean":"2023-10-06T14:26:22.885756+0000","last_became_active":"2023-10-06T14:22:39.418221+0000","last_became_peered":"2023-10-06T14:22:39.418221+0000","last_unstale":"2023-10-06T14:26:22.885756+0000","last_undegraded":"2023-10-06T14:26:22.885756+0000","last_fullsized":"2023-10-06T14:26:22.885756+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:46.111461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":920,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.866951+0000","last_change":"2023-10-06T14:22:39.210447+0000","last_active":"2023-10-06T14:26:22.866951+0000","last_peered":"2023-10-06T14:26:22.866951+0000","last_clean":"2023-10-06T14:26:22.866951+0000","last_became_active":"2023-10-06T14:22:39.195040+0000","last_became_peered":"2023-10-06T14:22:39.195040+0000","last_unstale":"2023-10-06T14:26:22.866951+0000","last_undegraded":"2023-10-06T14:26:22.866951+0000","last_fullsized":"2023-10-06T14:26:22.866951+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:29:47.926237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":925,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.729122+0000","last_change":"2023-10-06T14:16:33.305137+0000","last_active":"2023-10-06T14:26:20.729122+0000","last_peered":"2023-10-06T14:26:20.729122+0000","last_clean":"2023-10-06T14:26:20.729122+0000","last_became_active":"2023-10-06T14:16:33.300222+0000","last_became_peered":"2023-10-06T14:16:33.300222+0000","last_unstale":"2023-10-06T14:26:20.729122+0000","last_undegraded":"2023-10-06T14:26:20.729122+0000","last_fullsized":"2023-10-06T14:26:20.729122+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:34:52.750381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":919,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.822743+0000","last_change":"2023-10-06T14:22:39.745594+0000","last_active":"2023-10-06T14:26:22.822743+0000","last_peered":"2023-10-06T14:26:22.822743+0000","last_clean":"2023-10-06T14:26:22.822743+0000","last_became_active":"2023-10-06T14:22:39.712413+0000","last_became_peered":"2023-10-06T14:22:39.712413+0000","last_unstale":"2023-10-06T14:26:22.822743+0000","last_undegraded":"2023-10-06T14:26:22.822743+0000","last_fullsized":"2023-10-06T14:26:22.822743+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:59:53.774197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":905,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.206280+0000","last_change":"2023-10-06T13:54:19.475064+0000","last_active":"2023-10-06T14:26:21.206280+0000","last_peered":"2023-10-06T14:26:21.206280+0000","last_clean":"2023-10-06T14:26:21.206280+0000","last_became_active":"2023-10-06T13:54:19.427005+0000","last_became_peered":"2023-10-06T13:54:19.427005+0000","last_unstale":"2023-10-06T14:26:21.206280+0000","last_undegraded":"2023-10-06T14:26:21.206280+0000","last_fullsized":"2023-10-06T14:26:21.206280+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:21.985890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":910,"reported_epoch":968,"state":"active+clean","last_fresh":"2023-10-06T14:26:23.397157+0000","last_change":"2023-10-06T13:54:19.536231+0000","last_active":"2023-10-06T14:26:23.397157+0000","last_peered":"2023-10-06T14:26:23.397157+0000","last_clean":"2023-10-06T14:26:23.397157+0000","last_became_active":"2023-10-06T13:54:19.531077+0000","last_became_peered":"2023-10-06T13:54:19.531077+0000","last_unstale":"2023-10-06T14:26:23.397157+0000","last_undegraded":"2023-10-06T14:26:23.397157+0000","last_fullsized":"2023-10-06T14:26:23.397157+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:05.864633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":917,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.607069+0000","last_change":"2023-10-06T14:06:28.635986+0000","last_active":"2023-10-06T14:26:19.607069+0000","last_peered":"2023-10-06T14:26:19.607069+0000","last_clean":"2023-10-06T14:26:19.607069+0000","last_became_active":"2023-10-06T14:06:28.630618+0000","last_became_peered":"2023-10-06T14:06:28.630618+0000","last_unstale":"2023-10-06T14:26:19.607069+0000","last_undegraded":"2023-10-06T14:26:19.607069+0000","last_fullsized":"2023-10-06T14:26:19.607069+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:32.474746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":927,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.694758+0000","last_change":"2023-10-06T13:54:21.830131+0000","last_active":"2023-10-06T14:26:19.694758+0000","last_peered":"2023-10-06T14:26:19.694758+0000","last_clean":"2023-10-06T14:26:19.694758+0000","last_became_active":"2023-10-06T13:54:21.813915+0000","last_became_peered":"2023-10-06T13:54:21.813915+0000","last_unstale":"2023-10-06T14:26:19.694758+0000","last_undegraded":"2023-10-06T14:26:19.694758+0000","last_fullsized":"2023-10-06T14:26:19.694758+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:21:55.469659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":922,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.832923+0000","last_change":"2023-10-06T14:22:39.679043+0000","last_active":"2023-10-06T14:26:22.832923+0000","last_peered":"2023-10-06T14:26:22.832923+0000","last_clean":"2023-10-06T14:26:22.832923+0000","last_became_active":"2023-10-06T14:22:39.673285+0000","last_became_peered":"2023-10-06T14:22:39.673285+0000","last_unstale":"2023-10-06T14:26:22.832923+0000","last_undegraded":"2023-10-06T14:26:22.832923+0000","last_fullsized":"2023-10-06T14:26:22.832923+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:12.259360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":913,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.740540+0000","last_change":"2023-10-06T14:16:33.403287+0000","last_active":"2023-10-06T14:26:20.740540+0000","last_peered":"2023-10-06T14:26:20.740540+0000","last_clean":"2023-10-06T14:26:20.740540+0000","last_became_active":"2023-10-06T14:16:33.392050+0000","last_became_peered":"2023-10-06T14:16:33.392050+0000","last_unstale":"2023-10-06T14:26:20.740540+0000","last_undegraded":"2023-10-06T14:26:20.740540+0000","last_fullsized":"2023-10-06T14:26:20.740540+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:29:10.779601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":920,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.617510+0000","last_change":"2023-10-06T14:05:13.253436+0000","last_active":"2023-10-06T14:26:19.617510+0000","last_peered":"2023-10-06T14:26:19.617510+0000","last_clean":"2023-10-06T14:26:19.617510+0000","last_became_active":"2023-10-06T14:05:13.248166+0000","last_became_peered":"2023-10-06T14:05:13.248166+0000","last_unstale":"2023-10-06T14:26:19.617510+0000","last_undegraded":"2023-10-06T14:26:19.617510+0000","last_fullsized":"2023-10-06T14:26:19.617510+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:51:29.023023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":908,"reported_epoch":968,"state":"active+clean","last_fresh":"2023-10-06T14:26:23.401249+0000","last_change":"2023-10-06T13:54:26.224788+0000","last_active":"2023-10-06T14:26:23.401249+0000","last_peered":"2023-10-06T14:26:23.401249+0000","last_clean":"2023-10-06T14:26:23.401249+0000","last_became_active":"2023-10-06T13:54:26.218851+0000","last_became_peered":"2023-10-06T13:54:26.218851+0000","last_unstale":"2023-10-06T14:26:23.401249+0000","last_undegraded":"2023-10-06T14:26:23.401249+0000","last_fullsized":"2023-10-06T14:26:23.401249+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:43:20.812836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":939,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.838339+0000","last_change":"2023-10-06T14:22:39.234795+0000","last_active":"2023-10-06T14:26:22.838339+0000","last_peered":"2023-10-06T14:26:22.838339+0000","last_clean":"2023-10-06T14:26:22.838339+0000","last_became_active":"2023-10-06T14:22:39.109893+0000","last_became_peered":"2023-10-06T14:22:39.109893+0000","last_unstale":"2023-10-06T14:26:22.838339+0000","last_undegraded":"2023-10-06T14:26:22.838339+0000","last_fullsized":"2023-10-06T14:26:22.838339+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:27.039844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":925,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.735476+0000","last_change":"2023-10-06T14:16:33.559587+0000","last_active":"2023-10-06T14:26:20.735476+0000","last_peered":"2023-10-06T14:26:20.735476+0000","last_clean":"2023-10-06T14:26:20.735476+0000","last_became_active":"2023-10-06T14:16:33.551586+0000","last_became_peered":"2023-10-06T14:16:33.551586+0000","last_unstale":"2023-10-06T14:26:20.735476+0000","last_undegraded":"2023-10-06T14:26:20.735476+0000","last_fullsized":"2023-10-06T14:26:20.735476+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:53.251968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":918,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.741987+0000","last_change":"2023-10-06T14:16:33.406336+0000","last_active":"2023-10-06T14:26:20.741987+0000","last_peered":"2023-10-06T14:26:20.741987+0000","last_clean":"2023-10-06T14:26:20.741987+0000","last_became_active":"2023-10-06T14:16:33.398349+0000","last_became_peered":"2023-10-06T14:16:33.398349+0000","last_unstale":"2023-10-06T14:26:20.741987+0000","last_undegraded":"2023-10-06T14:26:20.741987+0000","last_fullsized":"2023-10-06T14:26:20.741987+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:24:58.230306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":928,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.738574+0000","last_change":"2023-10-06T13:54:31.926742+0000","last_active":"2023-10-06T14:26:22.738574+0000","last_peered":"2023-10-06T14:26:22.738574+0000","last_clean":"2023-10-06T14:26:22.738574+0000","last_became_active":"2023-10-06T13:54:31.908571+0000","last_became_peered":"2023-10-06T13:54:31.908571+0000","last_unstale":"2023-10-06T14:26:22.738574+0000","last_undegraded":"2023-10-06T14:26:22.738574+0000","last_fullsized":"2023-10-06T14:26:22.738574+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:45.581588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":928,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.868651+0000","last_change":"2023-10-06T14:22:39.277750+0000","last_active":"2023-10-06T14:26:22.868651+0000","last_peered":"2023-10-06T14:26:22.868651+0000","last_clean":"2023-10-06T14:26:22.868651+0000","last_became_active":"2023-10-06T14:22:39.078831+0000","last_became_peered":"2023-10-06T14:22:39.078831+0000","last_unstale":"2023-10-06T14:26:22.868651+0000","last_undegraded":"2023-10-06T14:26:22.868651+0000","last_fullsized":"2023-10-06T14:26:22.868651+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:52.651981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":916,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.595315+0000","last_change":"2023-10-06T14:05:13.235606+0000","last_active":"2023-10-06T14:26:19.595315+0000","last_peered":"2023-10-06T14:26:19.595315+0000","last_clean":"2023-10-06T14:26:19.595315+0000","last_became_active":"2023-10-06T14:05:13.230098+0000","last_became_peered":"2023-10-06T14:05:13.230098+0000","last_unstale":"2023-10-06T14:26:19.595315+0000","last_undegraded":"2023-10-06T14:26:19.595315+0000","last_fullsized":"2023-10-06T14:26:19.595315+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:28:31.276617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":928,"reported_epoch":968,"state":"active+clean","last_fresh":"2023-10-06T14:26:23.379085+0000","last_change":"2023-10-06T14:16:33.641985+0000","last_active":"2023-10-06T14:26:23.379085+0000","last_peered":"2023-10-06T14:26:23.379085+0000","last_clean":"2023-10-06T14:26:23.379085+0000","last_became_active":"2023-10-06T14:16:33.635681+0000","last_became_peered":"2023-10-06T14:16:33.635681+0000","last_unstale":"2023-10-06T14:26:23.379085+0000","last_undegraded":"2023-10-06T14:26:23.379085+0000","last_fullsized":"2023-10-06T14:26:23.379085+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:42.455510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":927,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.720626+0000","last_change":"2023-10-06T14:16:33.275781+0000","last_active":"2023-10-06T14:26:20.720626+0000","last_peered":"2023-10-06T14:26:20.720626+0000","last_clean":"2023-10-06T14:26:20.720626+0000","last_became_active":"2023-10-06T14:16:33.269648+0000","last_became_peered":"2023-10-06T14:16:33.269648+0000","last_unstale":"2023-10-06T14:26:20.720626+0000","last_undegraded":"2023-10-06T14:26:20.720626+0000","last_fullsized":"2023-10-06T14:26:20.720626+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:25.574583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":938,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.797168+0000","last_change":"2023-10-06T14:22:39.170180+0000","last_active":"2023-10-06T14:26:22.797168+0000","last_peered":"2023-10-06T14:26:22.797168+0000","last_clean":"2023-10-06T14:26:22.797168+0000","last_became_active":"2023-10-06T14:22:39.121176+0000","last_became_peered":"2023-10-06T14:22:39.121176+0000","last_unstale":"2023-10-06T14:26:22.797168+0000","last_undegraded":"2023-10-06T14:26:22.797168+0000","last_fullsized":"2023-10-06T14:26:22.797168+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:24.770331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":922,"reported_epoch":966,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.674582+0000","last_change":"2023-10-06T13:54:27.649937+0000","last_active":"2023-10-06T14:26:21.674582+0000","last_peered":"2023-10-06T14:26:21.674582+0000","last_clean":"2023-10-06T14:26:21.674582+0000","last_became_active":"2023-10-06T13:54:27.642873+0000","last_became_peered":"2023-10-06T13:54:27.642873+0000","last_unstale":"2023-10-06T14:26:21.674582+0000","last_undegraded":"2023-10-06T14:26:21.674582+0000","last_fullsized":"2023-10-06T14:26:21.674582+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:36:36.269635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":926,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.805049+0000","last_change":"2023-10-06T14:22:39.756236+0000","last_active":"2023-10-06T14:26:22.805049+0000","last_peered":"2023-10-06T14:26:22.805049+0000","last_clean":"2023-10-06T14:26:22.805049+0000","last_became_active":"2023-10-06T14:22:39.733473+0000","last_became_peered":"2023-10-06T14:22:39.733473+0000","last_unstale":"2023-10-06T14:26:22.805049+0000","last_undegraded":"2023-10-06T14:26:22.805049+0000","last_fullsized":"2023-10-06T14:26:22.805049+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:45:09.280243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":1053,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.740919+0000","last_change":"2023-10-06T14:16:33.639507+0000","last_active":"2023-10-06T14:26:22.740919+0000","last_peered":"2023-10-06T14:26:22.740919+0000","last_clean":"2023-10-06T14:26:22.740919+0000","last_became_active":"2023-10-06T14:16:33.632776+0000","last_became_peered":"2023-10-06T14:16:33.632776+0000","last_unstale":"2023-10-06T14:26:22.740919+0000","last_undegraded":"2023-10-06T14:26:22.740919+0000","last_fullsized":"2023-10-06T14:26:22.740919+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:08.156870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:08.156870+0000","last_clean_scrub_stamp":"2023-10-06T13:53:08.156870+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:08.683941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":908,"reported_epoch":968,"state":"active+clean","last_fresh":"2023-10-06T14:26:23.385649+0000","last_change":"2023-10-06T13:54:26.199772+0000","last_active":"2023-10-06T14:26:23.385649+0000","last_peered":"2023-10-06T14:26:23.385649+0000","last_clean":"2023-10-06T14:26:23.385649+0000","last_became_active":"2023-10-06T13:54:26.193354+0000","last_became_peered":"2023-10-06T13:54:26.193354+0000","last_unstale":"2023-10-06T14:26:23.385649+0000","last_undegraded":"2023-10-06T14:26:23.385649+0000","last_fullsized":"2023-10-06T14:26:23.385649+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:11.930838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":928,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.609432+0000","last_change":"2023-10-06T13:54:22.015064+0000","last_active":"2023-10-06T14:26:19.609432+0000","last_peered":"2023-10-06T14:26:19.609432+0000","last_clean":"2023-10-06T14:26:19.609432+0000","last_became_active":"2023-10-06T13:54:22.005243+0000","last_became_peered":"2023-10-06T13:54:22.005243+0000","last_unstale":"2023-10-06T14:26:19.609432+0000","last_undegraded":"2023-10-06T14:26:19.609432+0000","last_fullsized":"2023-10-06T14:26:19.609432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:46:20.602620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"43'7","reported_seq":935,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.193835+0000","last_change":"2023-10-06T14:05:13.464997+0000","last_active":"2023-10-06T14:26:21.193835+0000","last_peered":"2023-10-06T14:26:21.193835+0000","last_clean":"2023-10-06T14:26:21.193835+0000","last_became_active":"2023-10-06T14:05:13.458219+0000","last_became_peered":"2023-10-06T14:05:13.458219+0000","last_unstale":"2023-10-06T14:26:21.193835+0000","last_undegraded":"2023-10-06T14:26:21.193835+0000","last_fullsized":"2023-10-06T14:26:21.193835+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:40:26.823980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":928,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:20.694527+0000","last_change":"2023-10-06T14:22:39.278351+0000","last_active":"2023-10-06T14:26:20.694527+0000","last_peered":"2023-10-06T14:26:20.694527+0000","last_clean":"2023-10-06T14:26:20.694527+0000","last_became_active":"2023-10-06T14:22:39.269435+0000","last_became_peered":"2023-10-06T14:22:39.269435+0000","last_unstale":"2023-10-06T14:26:20.694527+0000","last_undegraded":"2023-10-06T14:26:20.694527+0000","last_fullsized":"2023-10-06T14:26:20.694527+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:10.817190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":921,"reported_epoch":968,"state":"active+clean","last_fresh":"2023-10-06T14:26:23.402751+0000","last_change":"2023-10-06T14:16:33.659581+0000","last_active":"2023-10-06T14:26:23.402751+0000","last_peered":"2023-10-06T14:26:23.402751+0000","last_clean":"2023-10-06T14:26:23.402751+0000","last_became_active":"2023-10-06T14:16:33.650127+0000","last_became_peered":"2023-10-06T14:16:33.650127+0000","last_unstale":"2023-10-06T14:26:23.402751+0000","last_undegraded":"2023-10-06T14:26:23.402751+0000","last_fullsized":"2023-10-06T14:26:23.402751+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:45.333800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":907,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.213411+0000","last_change":"2023-10-06T13:54:22.165757+0000","last_active":"2023-10-06T14:26:21.213411+0000","last_peered":"2023-10-06T14:26:21.213411+0000","last_clean":"2023-10-06T14:26:21.213411+0000","last_became_active":"2023-10-06T13:54:22.160421+0000","last_became_peered":"2023-10-06T13:54:22.160421+0000","last_unstale":"2023-10-06T14:26:21.213411+0000","last_undegraded":"2023-10-06T14:26:21.213411+0000","last_fullsized":"2023-10-06T14:26:21.213411+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:51.552552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":929,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.790789+0000","last_change":"2023-10-06T13:54:25.399248+0000","last_active":"2023-10-06T14:26:22.790789+0000","last_peered":"2023-10-06T14:26:22.790789+0000","last_clean":"2023-10-06T14:26:22.790789+0000","last_became_active":"2023-10-06T13:54:25.349894+0000","last_became_peered":"2023-10-06T13:54:25.349894+0000","last_unstale":"2023-10-06T14:26:22.790789+0000","last_undegraded":"2023-10-06T14:26:22.790789+0000","last_fullsized":"2023-10-06T14:26:22.790789+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:12:02.751197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":941,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.684113+0000","last_change":"2023-10-06T14:22:39.984989+0000","last_active":"2023-10-06T14:26:19.684113+0000","last_peered":"2023-10-06T14:26:19.684113+0000","last_clean":"2023-10-06T14:26:19.684113+0000","last_became_active":"2023-10-06T14:22:39.979167+0000","last_became_peered":"2023-10-06T14:22:39.979167+0000","last_unstale":"2023-10-06T14:26:19.684113+0000","last_undegraded":"2023-10-06T14:26:19.684113+0000","last_fullsized":"2023-10-06T14:26:19.684113+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":836,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:55:13.743000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":937,"reported_epoch":964,"state":"active+clean","last_fresh":"2023-10-06T14:26:19.635697+0000","last_change":"2023-10-06T14:05:13.200912+0000","last_active":"2023-10-06T14:26:19.635697+0000","last_peered":"2023-10-06T14:26:19.635697+0000","last_clean":"2023-10-06T14:26:19.635697+0000","last_became_active":"2023-10-06T14:05:13.191757+0000","last_became_peered":"2023-10-06T14:05:13.191757+0000","last_unstale":"2023-10-06T14:26:19.635697+0000","last_undegraded":"2023-10-06T14:26:19.635697+0000","last_fullsized":"2023-10-06T14:26:19.635697+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:21:31.319110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":906,"reported_epoch":965,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.187963+0000","last_change":"2023-10-06T13:54:22.153461+0000","last_active":"2023-10-06T14:26:21.187963+0000","last_peered":"2023-10-06T14:26:21.187963+0000","last_clean":"2023-10-06T14:26:21.187963+0000","last_became_active":"2023-10-06T13:54:22.144576+0000","last_became_peered":"2023-10-06T13:54:22.144576+0000","last_unstale":"2023-10-06T14:26:21.187963+0000","last_undegraded":"2023-10-06T14:26:21.187963+0000","last_fullsized":"2023-10-06T14:26:21.187963+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:55:12.616531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":927,"reported_epoch":967,"state":"active+clean","last_fresh":"2023-10-06T14:26:22.763619+0000","last_change":"2023-10-06T13:54:19.740406+0000","last_active":"2023-10-06T14:26:22.763619+0000","last_peered":"2023-10-06T14:26:22.763619+0000","last_clean":"2023-10-06T14:26:22.763619+0000","last_became_active":"2023-10-06T13:54:19.734381+0000","last_became_peered":"2023-10-06T13:54:19.734381+0000","last_unstale":"2023-10-06T14:26:22.763619+0000","last_undegraded":"2023-10-06T14:26:22.763619+0000","last_fullsized":"2023-10-06T14:26:22.763619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:23.226096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":934,"reported_epoch":966,"state":"active+clean","last_fresh":"2023-10-06T14:26:21.652071+0000","last_change":"2023-10-06T14:05:13.451797+0000","last_active":"2023-10-06T14:26:21.652071+0000","last_peered":"2023-10-06T14:26:21.652071+0000","last_clean":"2023-10-06T14:26:21.652071+0000","last_became_active":"2023-10-06T14:05:13.442612+0000","last_became_peered":"2023-10-06T14:05:13.442612+0000","last_unstale":"2023-10-06T14:26:21.652071+0000","last_undegraded":"2023-10-06T14:26:21.652071+0000","last_fullsized":"2023-10-06T14:26:21.652071+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:59:33.582696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719477122,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59108,"kb_used_data":5916,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312732,"statfs":{"total":96636764160,"available":96576237568,"internally_reserved":0,"allocated":6057984,"data_stored":3404017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":11.946,"5min":12.488,"15min":13.279},"min":{"1min":6.3920000000000003,"5min":4.6070000000000002,"15min":4.6070000000000002},"max":{"1min":52.079999999999998,"5min":91.744,"15min":109.691},"last":9.3650000000000002},{"interface":"front","average":{"1min":12.875999999999999,"5min":12.664,"15min":14.576000000000001},"min":{"1min":4.1870000000000003,"5min":4.1870000000000003,"15min":4.1870000000000003},"max":{"1min":44.232999999999997,"5min":67.936000000000007,"15min":166.33699999999999},"last":12.683999999999999}]},{"osd":2,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":12.112,"5min":11.595000000000001,"15min":13.598000000000001},"min":{"1min":7.9850000000000003,"5min":6.3920000000000003,"15min":4.9800000000000004},"max":{"1min":18.631,"5min":64.661000000000001,"15min":155.68799999999999},"last":24.128},{"interface":"front","average":{"1min":15.757999999999999,"5min":13.82,"15min":16.335000000000001},"min":{"1min":9.0660000000000007,"5min":7.3049999999999997,"15min":6.6059999999999999},"max":{"1min":47.368000000000002,"5min":92.335999999999999,"15min":212.16399999999999},"last":18.623999999999999}]},{"osd":3,"last update":"Fri Oct 6 14:25:44 2023","interfaces":[{"interface":"back","average":{"1min":23.273,"5min":29.661000000000001,"15min":33.582999999999998},"min":{"1min":9.6639999999999997,"5min":8.6850000000000005,"15min":8.6850000000000005},"max":{"1min":126.22799999999999,"5min":310.19,"15min":310.19},"last":13.875},{"interface":"front","average":{"1min":18.346,"5min":18.327999999999999,"15min":18.873999999999999},"min":{"1min":8.5370000000000008,"5min":7.8399999999999999,"15min":7.8399999999999999},"max":{"1min":65.375,"5min":65.375,"15min":65.375},"last":7.8600000000000003}]},{"osd":4,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":15290.522000000001,"5min":3070.721,"15min":1038.3879999999999},"min":{"1min":9.2720000000000002,"5min":7.8520000000000003,"15min":7.0190000000000001},"max":{"1min":39693.332999999999,"5min":39693.332999999999,"15min":39693.332999999999},"last":13.286},{"interface":"front","average":{"1min":15323.151,"5min":3078.1880000000001,"15min":1039.8889999999999},"min":{"1min":14.579000000000001,"5min":8.5679999999999996,"15min":7.3239999999999998},"max":{"1min":39697.402000000002,"5min":39697.402000000002,"15min":39697.402000000002},"last":11.196}]},{"osd":5,"last update":"Fri Oct 6 14:25:44 2023","interfaces":[{"interface":"back","average":{"1min":21.614999999999998,"5min":17.170000000000002,"15min":92.123000000000005},"min":{"1min":9.1199999999999992,"5min":9.1199999999999992,"15min":9.1199999999999992},"max":{"1min":69.561999999999998,"5min":69.561999999999998,"15min":8890.4069999999992},"last":17.754999999999999},{"interface":"front","average":{"1min":25.356000000000002,"5min":17.224,"15min":92.638999999999996},"min":{"1min":8.3140000000000001,"5min":7.8620000000000001,"15min":7.8620000000000001},"max":{"1min":139.78800000000001,"5min":139.78800000000001,"15min":8888.7939999999999},"last":14.494}]},{"osd":6,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.563000000000001,"5min":15.667999999999999,"15min":1233.97},"min":{"1min":9.9359999999999999,"5min":8.4710000000000001,"15min":7.3529999999999998},"max":{"1min":20.318000000000001,"5min":78.221999999999994,"15min":47304.226999999999},"last":20.283000000000001},{"interface":"front","average":{"1min":17.091000000000001,"5min":18.989000000000001,"15min":1237.328},"min":{"1min":11.202999999999999,"5min":10.666,"15min":8.9949999999999992},"max":{"1min":24.579000000000001,"5min":116.84999999999999,"15min":47302.955999999998},"last":16.690999999999999}]},{"osd":7,"last update":"Fri Oct 6 14:25:31 2023","interfaces":[{"interface":"back","average":{"1min":26.558,"5min":20.433,"15min":22.890000000000001},"min":{"1min":14.305999999999999,"5min":10.992000000000001,"15min":9.0250000000000004},"max":{"1min":78.769000000000005,"5min":78.769000000000005,"15min":214.02699999999999},"last":21.684999999999999},{"interface":"front","average":{"1min":30.707000000000001,"5min":20.494,"15min":22.446999999999999},"min":{"1min":11.108000000000001,"5min":9.8379999999999992,"15min":9.6159999999999997},"max":{"1min":134.17400000000001,"5min":134.17400000000001,"15min":201.52799999999999},"last":11.955}]}]},{"osd":0,"up_from":16,"seq":68719477138,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58716,"kb_used_data":5524,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313124,"statfs":{"total":96636764160,"available":96576638976,"internally_reserved":0,"allocated":5656576,"data_stored":3006177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":53,"apply_latency_ms":53,"commit_latency_ns":53000000,"apply_latency_ns":53000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 14:26:15 2023","interfaces":[{"interface":"back","average":{"1min":19.928999999999998,"5min":17.440999999999999,"15min":16.654},"min":{"1min":6.7800000000000002,"5min":5.4980000000000002,"15min":4.9720000000000004},"max":{"1min":97.855999999999995,"5min":135.13800000000001,"15min":342.61399999999998},"last":11.712},{"interface":"front","average":{"1min":21.222000000000001,"5min":19.111999999999998,"15min":17.628},"min":{"1min":5.9409999999999998,"5min":5.8810000000000002,"15min":5.5960000000000001},"max":{"1min":144.221,"5min":144.221,"15min":337.48000000000002},"last":13.032}]},{"osd":2,"last update":"Fri Oct 6 14:25:21 2023","interfaces":[{"interface":"back","average":{"1min":34.204999999999998,"5min":19.474,"15min":19.213000000000001},"min":{"1min":8.8699999999999992,"5min":7.2809999999999997,"15min":5.9669999999999996},"max":{"1min":184.036,"5min":184.036,"15min":184.036},"last":11.177},{"interface":"front","average":{"1min":30.013999999999999,"5min":15.800000000000001,"15min":17.484999999999999},"min":{"1min":5.718,"5min":5.718,"15min":5.718},"max":{"1min":181.06700000000001,"5min":181.06700000000001,"15min":181.06700000000001},"last":6.6470000000000002}]},{"osd":3,"last update":"Fri Oct 6 14:25:42 2023","interfaces":[{"interface":"back","average":{"1min":30.332000000000001,"5min":24.734999999999999,"15min":25.606999999999999},"min":{"1min":7.5949999999999998,"5min":7.4139999999999997,"15min":7.4139999999999997},"max":{"1min":189.10499999999999,"5min":189.10499999999999,"15min":189.10499999999999},"last":13.619999999999999},{"interface":"front","average":{"1min":31.155000000000001,"5min":24.603999999999999,"15min":24.998000000000001},"min":{"1min":10.960000000000001,"5min":6.9850000000000003,"15min":6.9850000000000003},"max":{"1min":163.60900000000001,"5min":163.60900000000001,"15min":163.60900000000001},"last":16.693999999999999}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":3655.643,"5min":746.56799999999998,"15min":262.995},"min":{"1min":13.273999999999999,"5min":8.7420000000000009,"15min":6.9269999999999996},"max":{"1min":39913.453999999998,"5min":39913.453999999998,"15min":39913.453999999998},"last":15.192},{"interface":"front","average":{"1min":3644.9029999999998,"5min":742.30499999999995,"15min":261.23700000000002},"min":{"1min":7.2050000000000001,"5min":7.2050000000000001,"15min":7.2050000000000001},"max":{"1min":39912.421999999999,"5min":39912.421999999999,"15min":39912.421999999999},"last":9.0779999999999994}]},{"osd":5,"last update":"Fri Oct 6 14:25:49 2023","interfaces":[{"interface":"back","average":{"1min":15.067,"5min":16.125,"15min":71.585999999999999},"min":{"1min":8.2599999999999998,"5min":7.5209999999999999,"15min":7.5209999999999999},"max":{"1min":56.817,"5min":142.267,"15min":6380.991},"last":17.486000000000001},{"interface":"front","average":{"1min":24.161000000000001,"5min":20.405000000000001,"15min":66.221000000000004},"min":{"1min":10.645,"5min":9.9390000000000001,"15min":9.4550000000000001},"max":{"1min":133.797,"5min":145.59299999999999,"15min":6266.0159999999996},"last":12.458}]},{"osd":6,"last update":"Fri Oct 6 14:26:05 2023","interfaces":[{"interface":"back","average":{"1min":13.15,"5min":20.032,"15min":725.85500000000002},"min":{"1min":7.7789999999999999,"5min":7.1479999999999997,"15min":7.1479999999999997},"max":{"1min":26.087,"5min":177.73699999999999,"15min":46416.196000000004},"last":10.26},{"interface":"front","average":{"1min":15.784000000000001,"5min":22.670000000000002,"15min":724.05899999999997},"min":{"1min":10.669,"5min":8.6609999999999996,"15min":6.9459999999999997},"max":{"1min":21.206,"5min":181.625,"15min":46403.283000000003},"last":18.282}]},{"osd":7,"last update":"Fri Oct 6 14:25:31 2023","interfaces":[{"interface":"back","average":{"1min":38.594000000000001,"5min":23.141999999999999,"15min":1331.846},"min":{"1min":14.388,"5min":10.557,"15min":9.8209999999999997},"max":{"1min":178.86500000000001,"5min":178.86500000000001,"15min":67028.107999999993},"last":14.598000000000001},{"interface":"front","average":{"1min":31.856999999999999,"5min":20.099,"15min":1329.4469999999999},"min":{"1min":9.3140000000000001,"5min":8.4320000000000004,"15min":8.4320000000000004},"max":{"1min":178.38900000000001,"5min":178.38900000000001,"15min":67001.737999999998},"last":9.6379999999999999}]}]},{"osd":4,"up_from":430,"seq":1846835937654,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58704,"kb_used_data":5512,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313136,"statfs":{"total":96636764160,"available":96576651264,"internally_reserved":0,"allocated":5644288,"data_stored":3017805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":61.039999999999999,"5min":21.318999999999999,"15min":15.34},"min":{"1min":7.0979999999999999,"5min":4.508,"15min":4.085},"max":{"1min":469.096,"5min":469.096,"15min":469.096},"last":7.2140000000000004},{"interface":"front","average":{"1min":60.287999999999997,"5min":20.288,"15min":17.068999999999999},"min":{"1min":8.5890000000000004,"5min":4.758,"15min":4.5999999999999996},"max":{"1min":423.42000000000002,"5min":423.42000000000002,"15min":423.42000000000002},"last":7.75}]},{"osd":1,"last update":"Fri Oct 6 14:26:13 2023","interfaces":[{"interface":"back","average":{"1min":81.888000000000005,"5min":27.257000000000001,"15min":19.550000000000001},"min":{"1min":7.3289999999999997,"5min":5.9850000000000003,"15min":4.9390000000000001},"max":{"1min":469.72800000000001,"5min":469.72800000000001,"15min":469.72800000000001},"last":10.218999999999999},{"interface":"front","average":{"1min":77.108999999999995,"5min":26.469999999999999,"15min":21.765999999999998},"min":{"1min":5.9320000000000004,"5min":5.9320000000000004,"15min":5.9320000000000004},"max":{"1min":425.584,"5min":425.584,"15min":425.584},"last":10.827999999999999}]},{"osd":2,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":113.371,"5min":34.213000000000001,"15min":23.704999999999998},"min":{"1min":7.6429999999999998,"5min":6.8159999999999998,"15min":6.4770000000000003},"max":{"1min":530.30399999999997,"5min":530.30399999999997,"15min":530.30399999999997},"last":12.532999999999999},{"interface":"front","average":{"1min":117.30200000000001,"5min":37.220999999999997,"15min":26.550999999999998},"min":{"1min":10.667999999999999,"5min":7.0750000000000002,"15min":6.5780000000000003},"max":{"1min":530.86099999999999,"5min":530.86099999999999,"15min":530.86099999999999},"last":13.74}]},{"osd":3,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":58.695,"5min":25.254999999999999,"15min":19.239000000000001},"min":{"1min":11.391,"5min":9.9220000000000006,"15min":9.9220000000000006},"max":{"1min":470.80099999999999,"5min":470.80099999999999,"15min":470.80099999999999},"last":15.529999999999999},{"interface":"front","average":{"1min":49.429000000000002,"5min":22.123999999999999,"15min":17.120000000000001},"min":{"1min":6.7489999999999997,"5min":6.7489999999999997,"15min":6.7489999999999997},"max":{"1min":422.88,"5min":422.88,"15min":422.88},"last":9.0169999999999995}]},{"osd":5,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":58.246000000000002,"5min":23.637,"15min":76.748000000000005},"min":{"1min":8.8409999999999993,"5min":8.2230000000000008,"15min":5.9509999999999996},"max":{"1min":431.57900000000001,"5min":431.57900000000001,"15min":8813.2510000000002},"last":11.443},{"interface":"front","average":{"1min":62.423000000000002,"5min":22.936,"15min":77.950000000000003},"min":{"1min":6.5590000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"max":{"1min":537.68399999999997,"5min":537.68399999999997,"15min":8811.7579999999998},"last":13.138}]},{"osd":6,"last update":"Fri Oct 6 14:26:13 2023","interfaces":[{"interface":"back","average":{"1min":96.150999999999996,"5min":31.527000000000001,"15min":920.85000000000002},"min":{"1min":9.7840000000000007,"5min":6.9470000000000001,"15min":6.9470000000000001},"max":{"1min":563.85900000000004,"5min":563.85900000000004,"15min":44817.595000000001},"last":14.313000000000001},{"interface":"front","average":{"1min":84.043000000000006,"5min":30.527999999999999,"15min":922.12},"min":{"1min":12.715,"5min":7.5830000000000002,"15min":7.3789999999999996},"max":{"1min":471.80700000000002,"5min":471.80700000000002,"15min":44816.752999999997},"last":14.970000000000001}]},{"osd":7,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":50.648000000000003,"5min":23.149999999999999,"15min":1630.123},"min":{"1min":9.8119999999999994,"5min":9.7119999999999997,"15min":7.569},"max":{"1min":564.822,"5min":564.822,"15min":65382.432000000001},"last":18.446000000000002},{"interface":"front","average":{"1min":45.387999999999998,"5min":21.542999999999999,"15min":1627.549},"min":{"1min":13.016999999999999,"5min":7.8639999999999999,"15min":7.4059999999999997},"max":{"1min":433.30799999999999,"5min":433.30799999999999,"15min":65371.873},"last":20.347999999999999}]}]},{"osd":3,"up_from":834,"seq":3582002725217,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58716,"kb_used_data":5524,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313124,"statfs":{"total":96636764160,"available":96576638976,"internally_reserved":0,"allocated":5656576,"data_stored":3006177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":82,"apply_latency_ms":82,"commit_latency_ns":82000000,"apply_latency_ns":82000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":9.8170000000000002,"5min":26.867000000000001,"15min":32.284999999999997},"min":{"1min":5.6029999999999998,"5min":4.8559999999999999,"15min":4.8559999999999999},"max":{"1min":18.390999999999998,"5min":188.56700000000001,"15min":188.56700000000001},"last":22.681000000000001},{"interface":"front","average":{"1min":16.754999999999999,"5min":27.206,"15min":31.43},"min":{"1min":5.9870000000000001,"5min":5.827,"15min":5.827},"max":{"1min":135.715,"5min":214.209,"15min":214.209},"last":38.872999999999998}]},{"osd":1,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":63.271000000000001,"5min":37.026000000000003,"15min":34.704000000000001},"min":{"1min":7.7329999999999997,"5min":7.4409999999999998,"15min":7.4409999999999998},"max":{"1min":993.702,"5min":993.702,"15min":993.702},"last":122.386},{"interface":"front","average":{"1min":65.965999999999994,"5min":38.597999999999999,"15min":36.668999999999997},"min":{"1min":7.8520000000000003,"5min":6.6100000000000003,"15min":6.6100000000000003},"max":{"1min":991.50599999999997,"5min":991.50599999999997,"15min":991.50599999999997},"last":147.75800000000001}]},{"osd":2,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":21.853000000000002,"5min":29.605,"15min":34.101999999999997},"min":{"1min":8.9570000000000007,"5min":8.4199999999999999,"15min":8.4199999999999999},"max":{"1min":154.76599999999999,"5min":230.25399999999999,"15min":230.25399999999999},"last":52.75},{"interface":"front","average":{"1min":19.876000000000001,"5min":27.256,"15min":30.832000000000001},"min":{"1min":8.1959999999999997,"5min":8.1959999999999997,"15min":8.1959999999999997},"max":{"1min":95.603999999999999,"5min":165.553,"15min":165.553},"last":34.790999999999997}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":2736.415,"5min":573.53200000000004,"15min":216.17099999999999},"min":{"1min":10.223000000000001,"5min":7.1020000000000003,"15min":7.1020000000000003},"max":{"1min":40828.069000000003,"5min":40828.069000000003,"15min":40828.069000000003},"last":126.91200000000001},{"interface":"front","average":{"1min":18.699999999999999,"5min":28.707999999999998,"15min":32.601999999999997},"min":{"1min":8.6150000000000002,"5min":8.6150000000000002,"15min":8.6150000000000002},"max":{"1min":100.607,"5min":194.14599999999999,"15min":194.14599999999999},"last":83.572999999999993}]},{"osd":5,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":19.344000000000001,"5min":30.407,"15min":34.871000000000002},"min":{"1min":10.394,"5min":10.394,"15min":10.394},"max":{"1min":98.769000000000005,"5min":196.50200000000001,"15min":196.50200000000001},"last":36.985999999999997},{"interface":"front","average":{"1min":18.803000000000001,"5min":27.030999999999999,"15min":30.731000000000002},"min":{"1min":7.9329999999999998,"5min":7.9329999999999998,"15min":7.9329999999999998},"max":{"1min":111.729,"5min":177.739,"15min":177.739},"last":42.231999999999999}]},{"osd":6,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":20.492999999999999,"5min":38.835999999999999,"15min":47.399000000000001},"min":{"1min":9.6929999999999996,"5min":9.6929999999999996,"15min":9.6929999999999996},"max":{"1min":113.20399999999999,"5min":381.13,"15min":381.13},"last":127.839},{"interface":"front","average":{"1min":22.385999999999999,"5min":32.366999999999997,"15min":37.090000000000003},"min":{"1min":10.928000000000001,"5min":10.314,"15min":10.314},"max":{"1min":108.327,"5min":245.16200000000001,"15min":245.16200000000001},"last":109.922}]},{"osd":7,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":23.408999999999999,"5min":27.462,"15min":29.341999999999999},"min":{"1min":11.864000000000001,"5min":11.411,"15min":11.411},"max":{"1min":113.56699999999999,"5min":227.42599999999999,"15min":227.42599999999999},"last":81.742999999999995},{"interface":"front","average":{"1min":21.158000000000001,"5min":24.553999999999998,"15min":25.303999999999998},"min":{"1min":11.800000000000001,"5min":11.800000000000001,"15min":11.800000000000001},"max":{"1min":101.83499999999999,"5min":164.09899999999999,"15min":164.09899999999999},"last":52.259999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509831,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58660,"kb_used_data":5468,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313180,"statfs":{"total":96636764160,"available":96576696320,"internally_reserved":0,"allocated":5599232,"data_stored":2976840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":12.117000000000001,"5min":11.677,"15min":16.064},"min":{"1min":7.3819999999999997,"5min":5.452,"15min":5.3630000000000004},"max":{"1min":24.227,"5min":31.484000000000002,"15min":182.96600000000001},"last":37.688000000000002},{"interface":"front","average":{"1min":10.268000000000001,"5min":12.111000000000001,"15min":17.263000000000002},"min":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"max":{"1min":17.821000000000002,"5min":44.423000000000002,"15min":226.68600000000001},"last":27.803000000000001}]},{"osd":1,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":18.187000000000001,"5min":16.309999999999999,"15min":17.614000000000001},"min":{"1min":7.6710000000000003,"5min":7.3630000000000004,"15min":6.0970000000000004},"max":{"1min":100.065,"5min":101.874,"15min":148.20400000000001},"last":38.289000000000001},{"interface":"front","average":{"1min":12.387,"5min":14.256,"15min":17.227},"min":{"1min":6.6840000000000002,"5min":6.4630000000000001,"15min":6.3140000000000001},"max":{"1min":53.959000000000003,"5min":161.97499999999999,"15min":229.82499999999999},"last":51.273000000000003}]},{"osd":3,"last update":"Fri Oct 6 14:25:48 2023","interfaces":[{"interface":"back","average":{"1min":16.808,"5min":16.744,"15min":16.396000000000001},"min":{"1min":8.2769999999999992,"5min":8.2769999999999992,"15min":8.2769999999999992},"max":{"1min":31.460000000000001,"5min":33.399999999999999,"15min":33.399999999999999},"last":62.646999999999998},{"interface":"front","average":{"1min":13.861000000000001,"5min":14.6,"15min":14.781000000000001},"min":{"1min":8.6799999999999997,"5min":8.2509999999999994,"15min":8.2509999999999994},"max":{"1min":28.713000000000001,"5min":29.898,"15min":29.898},"last":64.054000000000002}]},{"osd":4,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":14883.07,"5min":2990.154,"15min":1010.401},"min":{"1min":6.8150000000000004,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":38405.120000000003,"5min":38405.120000000003,"15min":38405.120000000003},"last":70.638000000000005},{"interface":"front","average":{"1min":14785.589,"5min":2970.7600000000002,"15min":1006.212},"min":{"1min":7.8520000000000003,"5min":7.6550000000000002,"15min":7.6550000000000002},"max":{"1min":38264.116000000002,"5min":38264.116000000002,"15min":38264.116000000002},"last":28.445}]},{"osd":5,"last update":"Fri Oct 6 14:25:51 2023","interfaces":[{"interface":"back","average":{"1min":16.574999999999999,"5min":16.280999999999999,"15min":92.066999999999993},"min":{"1min":8.2560000000000002,"5min":8.2560000000000002,"15min":7.6589999999999998},"max":{"1min":32.328000000000003,"5min":50.210000000000001,"15min":9721.0490000000009},"last":73.159000000000006},{"interface":"front","average":{"1min":14.137,"5min":16.710999999999999,"15min":96.823999999999998},"min":{"1min":8.6099999999999994,"5min":8.6099999999999994,"15min":8.6099999999999994},"max":{"1min":24.001000000000001,"5min":73.320999999999998,"15min":9793.7890000000007},"last":77.673000000000002}]},{"osd":6,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":14.999000000000001,"5min":17.789000000000001,"15min":1309.6320000000001},"min":{"1min":8.8689999999999998,"5min":8.8689999999999998,"15min":8.8689999999999998},"max":{"1min":23.431999999999999,"5min":44.606999999999999,"15min":44531.741000000002},"last":34.725999999999999},{"interface":"front","average":{"1min":16.434000000000001,"5min":16.972000000000001,"15min":1305.6900000000001},"min":{"1min":7.5330000000000004,"5min":7.141,"15min":7.141},"max":{"1min":49.576999999999998,"5min":49.576999999999998,"15min":44501.423000000003},"last":76.936000000000007}]},{"osd":7,"last update":"Fri Oct 6 14:25:34 2023","interfaces":[{"interface":"back","average":{"1min":20.236000000000001,"5min":20.585000000000001,"15min":1733.5730000000001},"min":{"1min":12.177,"5min":10.631,"15min":8.6189999999999998},"max":{"1min":38.399999999999999,"5min":61.658999999999999,"15min":69773.953999999998},"last":73.903000000000006},{"interface":"front","average":{"1min":21.065999999999999,"5min":18.300999999999998,"15min":1723.3030000000001},"min":{"1min":11.403,"5min":9.5909999999999993,"15min":8.5429999999999993},"max":{"1min":31.567,"5min":34.923000000000002,"15min":69547.547999999995},"last":78.433999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62840,"kb_used_data":5552,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94309000,"statfs":{"total":96636764160,"available":96572416000,"internally_reserved":0,"allocated":5685248,"data_stored":3054367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":56,"apply_latency_ms":56,"commit_latency_ns":56000000,"apply_latency_ns":56000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":8.6549999999999994,"5min":11.162000000000001,"15min":18.341000000000001},"min":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.4050000000000002},"max":{"1min":12.99,"5min":94.040999999999997,"15min":574.78800000000001},"last":5.0999999999999996},{"interface":"front","average":{"1min":11.332000000000001,"5min":15.573,"15min":21.341999999999999},"min":{"1min":7.5309999999999997,"5min":7.5309999999999997,"15min":4.5140000000000002},"max":{"1min":17.783999999999999,"5min":130.036,"15min":522.72699999999998},"last":7.6779999999999999}]},{"osd":1,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.231,"5min":27.030999999999999,"15min":24.16},"min":{"1min":7.3150000000000004,"5min":5.4059999999999997,"15min":5.4059999999999997},"max":{"1min":14.238,"5min":1271.6179999999999,"15min":1271.6179999999999},"last":9.1989999999999998},{"interface":"front","average":{"1min":12.869,"5min":30.803000000000001,"15min":27.870999999999999},"min":{"1min":9.9900000000000002,"5min":6.8639999999999999,"15min":6.8639999999999999},"max":{"1min":17.446999999999999,"5min":1322.71,"15min":1322.71},"last":12.44}]},{"osd":2,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.842000000000001,"5min":14.388999999999999,"15min":19.920999999999999},"min":{"1min":9.4809999999999999,"5min":6.1660000000000004,"15min":5.532},"max":{"1min":30.937999999999999,"5min":101.197,"15min":525.81600000000003},"last":16.143999999999998},{"interface":"front","average":{"1min":11.125999999999999,"5min":17.469999999999999,"15min":24.055},"min":{"1min":6.7409999999999997,"5min":6.4509999999999996,"15min":6.4509999999999996},"max":{"1min":27.02,"5min":99.492999999999995,"15min":468.70400000000001},"last":9.9740000000000002}]},{"osd":3,"last update":"Fri Oct 6 14:25:41 2023","interfaces":[{"interface":"back","average":{"1min":15.800000000000001,"5min":20.021000000000001,"15min":20.113},"min":{"1min":6.8470000000000004,"5min":6.8470000000000004,"15min":6.8470000000000004},"max":{"1min":40.512,"5min":126.55500000000001,"15min":126.55500000000001},"last":17.16},{"interface":"front","average":{"1min":14.677,"5min":21.684999999999999,"15min":23.826000000000001},"min":{"1min":6.0389999999999997,"5min":6.0389999999999997,"15min":6.0389999999999997},"max":{"1min":49.473999999999997,"5min":236.27000000000001,"15min":236.27000000000001},"last":11.718}]},{"osd":4,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15821.066000000001,"5min":3179.7730000000001,"15min":1076.7719999999999},"min":{"1min":10.141,"5min":9.1189999999999998,"15min":8.7490000000000006},"max":{"1min":38720.860999999997,"5min":38720.860999999997,"15min":38720.860999999997},"last":14.757},{"interface":"front","average":{"1min":15816.687,"5min":3177.5610000000001,"15min":1074.1969999999999},"min":{"1min":6.782,"5min":6.782,"15min":6.782},"max":{"1min":38719.173000000003,"5min":38719.173000000003,"15min":38719.173000000003},"last":13.971}]},{"osd":5,"last update":"Fri Oct 6 14:25:51 2023","interfaces":[{"interface":"back","average":{"1min":17.318999999999999,"5min":21.241,"15min":107.18600000000001},"min":{"1min":8.8800000000000008,"5min":8.8800000000000008,"15min":8.8800000000000008},"max":{"1min":43.317999999999998,"5min":219.06200000000001,"15min":8032.4189999999999},"last":11.210000000000001},{"interface":"front","average":{"1min":18.963999999999999,"5min":21.207999999999998,"15min":109.759},"min":{"1min":7.4409999999999998,"5min":7.1459999999999999,"15min":7.1459999999999999},"max":{"1min":69.081000000000003,"5min":171.041,"15min":8030.5749999999998},"last":16.884}]},{"osd":7,"last update":"Fri Oct 6 14:25:37 2023","interfaces":[{"interface":"back","average":{"1min":20.959,"5min":19.791,"15min":1130.614},"min":{"1min":11.01,"5min":11.01,"15min":11.01},"max":{"1min":45.648000000000003,"5min":93.355999999999995,"15min":69750.712},"last":18.297000000000001},{"interface":"front","average":{"1min":16.279,"5min":14.369999999999999,"15min":1124.646},"min":{"1min":7.1769999999999996,"5min":6.1070000000000002,"15min":5.984},"max":{"1min":40.57,"5min":121.146,"15min":69749.794999999998},"last":10.629}]}]},{"osd":5,"up_from":659,"seq":2830383448439,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77156,"kb_used_data":5916,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94294684,"statfs":{"total":96636764160,"available":96557756416,"internally_reserved":0,"allocated":6057984,"data_stored":3418788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":65,"apply_latency_ms":65,"commit_latency_ns":65000000,"apply_latency_ns":65000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":10.645,"5min":13.339,"15min":38.990000000000002},"min":{"1min":4.6189999999999998,"5min":4.3339999999999996,"15min":4.1879999999999997},"max":{"1min":22.547999999999998,"5min":173.54599999999999,"15min":680.28200000000004},"last":7.141},{"interface":"front","average":{"1min":7.4779999999999998,"5min":11.548999999999999,"15min":29.652000000000001},"min":{"1min":4.7430000000000003,"5min":4.5579999999999998,"15min":4.5579999999999998},"max":{"1min":16.622,"5min":154.36199999999999,"15min":608.46199999999999},"last":14.422000000000001}]},{"osd":1,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":45.543999999999997,"5min":20.469000000000001,"15min":43.674999999999997},"min":{"1min":6.3620000000000001,"5min":5.4930000000000003,"15min":5.4930000000000003},"max":{"1min":633.78700000000003,"5min":633.78700000000003,"15min":633.78700000000003},"last":12.051},{"interface":"front","average":{"1min":46.32,"5min":23.73,"15min":46.381},"min":{"1min":10.032,"5min":5.9000000000000004,"15min":5.8259999999999996},"max":{"1min":598.60000000000002,"5min":598.60000000000002,"15min":701.19399999999996},"last":11.423}]},{"osd":2,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":14.177,"5min":14.871,"15min":44.731999999999999},"min":{"1min":8.218,"5min":6.8369999999999997,"15min":6.6029999999999998},"max":{"1min":27.632000000000001,"5min":87.468000000000004,"15min":603.024},"last":16.187999999999999},{"interface":"front","average":{"1min":15.343,"5min":17.352,"15min":49.799999999999997},"min":{"1min":10.210000000000001,"5min":8.1289999999999996,"15min":8.1289999999999996},"max":{"1min":29.491,"5min":84.820999999999998,"15min":690.71799999999996},"last":19.492000000000001}]},{"osd":3,"last update":"Fri Oct 6 14:25:42 2023","interfaces":[{"interface":"back","average":{"1min":15.289999999999999,"5min":28.317,"15min":32.777999999999999},"min":{"1min":9.0920000000000005,"5min":8.1690000000000005,"15min":8.1690000000000005},"max":{"1min":31.006,"5min":264.37900000000002,"15min":264.37900000000002},"last":20.670000000000002},{"interface":"front","average":{"1min":16.370000000000001,"5min":27.132999999999999,"15min":30.166},"min":{"1min":11.48,"5min":9.0990000000000002,"15min":9.0990000000000002},"max":{"1min":33.726999999999997,"5min":173.858,"15min":173.858},"last":18.087}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":3491.913,"5min":712.81600000000003,"15min":274.36799999999999},"min":{"1min":10.141,"5min":7.9340000000000002,"15min":7.9340000000000002},"max":{"1min":38256.345000000001,"5min":38256.345000000001,"15min":38256.345000000001},"last":22.091999999999999},{"interface":"front","average":{"1min":3489.7570000000001,"5min":714.00800000000004,"15min":281.79899999999998},"min":{"1min":13.077999999999999,"5min":9.1750000000000007,"15min":8.1839999999999993},"max":{"1min":38222.745999999999,"5min":38222.745999999999,"15min":38222.745999999999},"last":12.853}]},{"osd":6,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":15.797000000000001,"5min":18.550999999999998,"15min":47.345999999999997},"min":{"1min":10.766,"5min":7.7300000000000004,"15min":7.7300000000000004},"max":{"1min":27.385000000000002,"5min":108.053,"15min":607.87400000000002},"last":16.995999999999999},{"interface":"front","average":{"1min":17.838000000000001,"5min":21.890000000000001,"15min":51.551000000000002},"min":{"1min":14.509,"5min":10.855,"15min":7.1040000000000001},"max":{"1min":26.152000000000001,"5min":91.733000000000004,"15min":739.33100000000002},"last":20.09}]},{"osd":7,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":17.960999999999999,"5min":21.77,"15min":47.536000000000001},"min":{"1min":10.858000000000001,"5min":9.3569999999999993,"15min":7.0350000000000001},"max":{"1min":32.581000000000003,"5min":90.037000000000006,"15min":609.59799999999996},"last":15.159000000000001},{"interface":"front","average":{"1min":9.0069999999999997,"5min":13.292,"15min":44.540999999999997},"min":{"1min":6.4420000000000002,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":13.343999999999999,"5min":77.034999999999997,"15min":610.43799999999999},"last":10.16}]}]},{"osd":7,"up_from":13,"seq":55834575233,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80860,"kb_used_data":5524,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94290980,"statfs":{"total":96636764160,"available":96553963520,"internally_reserved":0,"allocated":5656576,"data_stored":3006177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:25:26 2023","interfaces":[{"interface":"back","average":{"1min":8.4930000000000003,"5min":10.589,"15min":33.819000000000003},"min":{"1min":4.907,"5min":4.742,"15min":4.2539999999999996},"max":{"1min":14.169,"5min":46.529000000000003,"15min":930.62099999999998},"last":8.2639999999999993},{"interface":"front","average":{"1min":14.048,"5min":11.707000000000001,"15min":31.795999999999999},"min":{"1min":6.6950000000000003,"5min":4.8079999999999998,"15min":4.8079999999999998},"max":{"1min":69.477999999999994,"5min":69.477999999999994,"15min":879.35199999999998},"last":7.601}]},{"osd":1,"last update":"Fri Oct 6 14:25:26 2023","interfaces":[{"interface":"back","average":{"1min":70.927000000000007,"5min":26.693999999999999,"15min":40.520000000000003},"min":{"1min":5.4790000000000001,"5min":5.4790000000000001,"15min":5.4790000000000001},"max":{"1min":1094.5889999999999,"5min":1094.5889999999999,"15min":1094.5889999999999},"last":11.683},{"interface":"front","average":{"1min":69.433000000000007,"5min":25.927,"15min":25.18},"min":{"1min":8.3170000000000002,"5min":6.2270000000000003,"15min":5.4539999999999997},"max":{"1min":1057.856,"5min":1057.856,"15min":1057.856},"last":14.989000000000001}]},{"osd":2,"last update":"Fri Oct 6 14:25:26 2023","interfaces":[{"interface":"back","average":{"1min":16.998000000000001,"5min":15.606999999999999,"15min":24.332000000000001},"min":{"1min":9.0139999999999993,"5min":6.5149999999999997,"15min":6.202},"max":{"1min":75.614000000000004,"5min":75.823999999999998,"15min":316.84399999999999},"last":6.9969999999999999},{"interface":"front","average":{"1min":18.135999999999999,"5min":16.129999999999999,"15min":40.203000000000003},"min":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"max":{"1min":78.471999999999994,"5min":98.183000000000007,"15min":931.94500000000005},"last":9.9499999999999993}]},{"osd":3,"last update":"Fri Oct 6 14:25:47 2023","interfaces":[{"interface":"back","average":{"1min":14.034000000000001,"5min":20.844000000000001,"15min":22.948},"min":{"1min":6.9969999999999999,"5min":6.9969999999999999,"15min":6.9969999999999999},"max":{"1min":40.936,"5min":184.93700000000001,"15min":184.93700000000001},"last":10.542999999999999},{"interface":"front","average":{"1min":13.778,"5min":18.068999999999999,"15min":18.817},"min":{"1min":8.2010000000000005,"5min":7.6699999999999999,"15min":7.6699999999999999},"max":{"1min":32.956000000000003,"5min":77.144000000000005,"15min":77.144000000000005},"last":15.557}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":2132.1480000000001,"5min":439.87099999999998,"15min":167.66999999999999},"min":{"1min":7.8869999999999996,"5min":5.9429999999999996,"15min":5.9429999999999996},"max":{"1min":40210.642,"5min":40210.642,"15min":40210.642},"last":9.2669999999999995},{"interface":"front","average":{"1min":2144.9340000000002,"5min":442.64400000000001,"15min":163.51599999999999},"min":{"1min":11.497999999999999,"5min":7.4180000000000001,"15min":6.9279999999999999},"max":{"1min":40377.220000000001,"5min":40377.220000000001,"15min":40377.220000000001},"last":17.106999999999999}]},{"osd":5,"last update":"Fri Oct 6 14:25:50 2023","interfaces":[{"interface":"back","average":{"1min":17.524000000000001,"5min":18.869,"15min":51.777000000000001},"min":{"1min":8.4939999999999998,"5min":8.1229999999999993,"15min":7.6909999999999998},"max":{"1min":70.796000000000006,"5min":149.61799999999999,"15min":6750.3770000000004},"last":11.081},{"interface":"front","average":{"1min":19.053999999999998,"5min":19.542000000000002,"15min":52.655000000000001},"min":{"1min":7.9029999999999996,"5min":6.9290000000000003,"15min":6.9290000000000003},"max":{"1min":83.019000000000005,"5min":174.839,"15min":6875.5780000000004},"last":16.416}]},{"osd":6,"last update":"Fri Oct 6 14:25:26 2023","interfaces":[{"interface":"back","average":{"1min":20.823,"5min":17.948,"15min":27.071999999999999},"min":{"1min":11.436999999999999,"5min":7.9509999999999996,"15min":7.7290000000000001},"max":{"1min":75.013999999999996,"5min":75.013999999999996,"15min":380.74000000000001},"last":14.266},{"interface":"front","average":{"1min":23.314,"5min":19.940999999999999,"15min":44.384999999999998},"min":{"1min":9.1120000000000001,"5min":8.9489999999999998,"15min":8.9489999999999998},"max":{"1min":83.512,"5min":120.83799999999999,"15min":935.49300000000005},"last":12.444000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T14:26:26.896 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T14:26:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.999+0000 7f9141a7d640 10 monclient: get_monmap_and_config 2023-10-06T14:26:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.999+0000 7f9141a7d640 10 monclient: build_initial_monmap 2023-10-06T14:26:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:26.999+0000 7f9141a7d640 10 monclient: monmap: 2023-10-06T14:26:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:27.000658+0000 2023-10-06T14:26:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:27.000658+0000 2023-10-06T14:26:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:27.003 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:27.003 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:27.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.000+0000 7f9141a7d640 1 Processor -- start 2023-10-06T14:26:27.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.000+0000 7f9141a7d640 1 -- start start 2023-10-06T14:26:27.004 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.000+0000 7f9141a7d640 10 monclient: init 2023-10-06T14:26:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.000+0000 7f9141a7d640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.000+0000 7f9141a7d640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2023-10-06T14:26:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.000+0000 7f9141a7d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f913c06b6d0 addr v1:172.21.15.19:6789/0 2023-10-06T14:26:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.000+0000 7f9141a7d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f913c144910 0x7f913c13df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.000+0000 7f9141a7d640 10 monclient(hunting): picked mon.noname-b con 0x7f913c144910 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.000+0000 7f9141a7d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f913c13e460 0x7f913c14b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.000+0000 7f9141a7d640 10 monclient(hunting): picked mon.noname-a con 0x7f913c13e460 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.000+0000 7f9141a7d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913c06baa0 con 0x7f913c06b6d0 2023-10-06T14:26:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.001+0000 7f9141a7d640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.001+0000 7f9141a7d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913c0b2530 con 0x7f913c13e460 2023-10-06T14:26:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.001+0000 7f9141a7d640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.001+0000 7f9141a7d640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0b2810 con 0x7f913c144910 2023-10-06T14:26:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.001+0000 7f9141a7d640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.001+0000 7f9141a7d640 10 monclient(hunting): authenticate will time out at 3315.187376s 2023-10-06T14:26:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.001+0000 7f913affd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f913c144910 0x7f913c13df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.001+0000 7f913bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f913c13e460 0x7f913c14b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.002+0000 7f913affd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f913c144910 0x7f913c13df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40230/0 (socket says 172.21.15.19:40230) 2023-10-06T14:26:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.002+0000 7f913affd640 1 -- 172.21.15.19:0/879298585 learned_addr learned my addr 172.21.15.19:0/879298585 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.002+0000 7f913affd640 10 monclient(hunting): get_auth_request con 0x7f913c144910 auth_method 0 2023-10-06T14:26:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.002+0000 7f913affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.002+0000 7f913affd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.002+0000 7f913affd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.002+0000 7f913bfff640 10 monclient(hunting): get_auth_request con 0x7f913c13e460 auth_method 0 2023-10-06T14:26:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.002+0000 7f913bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.002+0000 7f913bfff640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.002+0000 7f913bfff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.002+0000 7f913affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.002+0000 7f913affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.002+0000 7f913affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.003+0000 7f913bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.003+0000 7f913bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.003+0000 7f913bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.004+0000 7f913affd640 10 monclient(hunting): handle_auth_done global_id 17126 payload 274 2023-10-06T14:26:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.004+0000 7f913affd640 1 -- 172.21.15.19:0/879298585 >> v2:172.21.15.19:3300/0 conn(0x7f913c13e460 msgr2=0x7f913c14b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.004+0000 7f913affd640 1 --2- 172.21.15.19:0/879298585 >> v2:172.21.15.19:3300/0 conn(0x7f913c13e460 0x7f913c14b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.004+0000 7f913affd640 1 -- 172.21.15.19:0/879298585 >> v1:172.21.15.19:6789/0 conn(0x7f913c06b6d0 legacy=0x7f913c144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.004+0000 7f913affd640 10 monclient: _finish_hunting 0 2023-10-06T14:26:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.004+0000 7f913affd640 1 monclient: found mon.noname-b 2023-10-06T14:26:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.004+0000 7f913affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.004+0000 7f913affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T14:26:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.004+0000 7f913affd640 1 -- 172.21.15.19:0/879298585 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c0aeff0 con 0x7f913c144910 2023-10-06T14:26:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.006+0000 7f913affd640 1 --2- 172.21.15.19:0/879298585 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f913c144910 0x7f913c13df20 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 crypto rx=0x7f912c009870 tx=0x7f912c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66bba648b856dc14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f913a7fc640 1 -- 172.21.15.19:0/879298585 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c030840 con 0x7f913c144910 2023-10-06T14:26:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f913a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f913a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f913a7fc640 10 monclient: dump: 2023-10-06T14:26:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:27.014 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f913a7fc640 10 monclient: _finish_auth 0 2023-10-06T14:26:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f913a7fc640 10 monclient: _check_auth_tickets 2023-10-06T14:26:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f913a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f913a7fc640 1 -- 172.21.15.19:0/879298585 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c0309a0 con 0x7f913c144910 2023-10-06T14:26:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f9141a7d640 5 monclient: authenticate success, global_id 17126 2023-10-06T14:26:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f9141a7d640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f913a7fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f9141a7d640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f9141a7d640 4 set_mon_vals no callback set 2023-10-06T14:26:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f913a7fc640 1 -- 172.21.15.19:0/879298585 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c030d30 con 0x7f913c144910 2023-10-06T14:26:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f913a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f913a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.010+0000 7f913a7fc640 10 monclient: dump: 2023-10-06T14:26:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:27.016 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:27.016 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:27.016 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.011+0000 7f9141a7d640 10 monclient: shutdown 2023-10-06T14:26:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.011+0000 7f9141a7d640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.011+0000 7f9141a7d640 1 -- 172.21.15.19:0/879298585 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f913c144910 msgr2=0x7f913c13df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.011+0000 7f9141a7d640 1 --2- 172.21.15.19:0/879298585 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f913c144910 0x7f913c13df20 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 crypto rx=0x7f912c009870 tx=0x7f912c02b980 comp rx=0 tx=0).stop 2023-10-06T14:26:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.011+0000 7f9141a7d640 1 -- 172.21.15.19:0/879298585 shutdown_connections 2023-10-06T14:26:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.011+0000 7f9141a7d640 1 --2- 172.21.15.19:0/879298585 >> v2:172.21.15.19:3300/0 conn(0x7f913c13e460 0x7f913c14b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.011+0000 7f9141a7d640 1 --2- 172.21.15.19:0/879298585 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f913c144910 0x7f913c13df20 unknown :-1 s=CLOSED pgs=3119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.011+0000 7f9141a7d640 1 -- 172.21.15.19:0/879298585 >> 172.21.15.19:0/879298585 conn(0x7f913c0665b0 msgr2=0x7f913c06ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.011+0000 7f9141a7d640 1 -- 172.21.15.19:0/879298585 shutdown_connections 2023-10-06T14:26:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.011+0000 7f9141a7d640 1 -- 172.21.15.19:0/879298585 wait complete. 2023-10-06T14:26:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.012+0000 7f9141a7d640 10 monclient: build_initial_monmap 2023-10-06T14:26:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.012+0000 7f9141a7d640 10 monclient: monmap: 2023-10-06T14:26:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:27.021 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.012+0000 7f9141a7d640 1 librados: starting msgr at 2023-10-06T14:26:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.012+0000 7f9141a7d640 1 librados: starting objecter 2023-10-06T14:26:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.012+0000 7f9141a7d640 1 Processor -- start 2023-10-06T14:26:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.012+0000 7f9141a7d640 1 -- start start 2023-10-06T14:26:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.012+0000 7f9141a7d640 1 librados: setting wanted keys 2023-10-06T14:26:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.012+0000 7f9141a7d640 1 librados: calling monclient init 2023-10-06T14:26:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.012+0000 7f9141a7d640 10 monclient: init 2023-10-06T14:26:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T14:26:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f913c06b6d0 0x7f913c141620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 10 monclient(hunting): picked mon.noname-a con 0x7f913c06b6d0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f913c13e460 0x7f913c0a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 10 monclient(hunting): picked mon.noname-b con 0x7f913c13e460 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f913c144910 0x7f913c1383a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 10 monclient(hunting): picked mon.noname-c con 0x7f913c144910 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0a8c20 con 0x7f913c06b6d0 2023-10-06T14:26:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c140790 con 0x7f913c13e460 2023-10-06T14:26:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0a8ee0 con 0x7f913c144910 2023-10-06T14:26:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f9141a7d640 10 monclient(hunting): authenticate will time out at 3315.199823s 2023-10-06T14:26:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f913bfff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f913c144910 0x7f913c1383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.013+0000 7f913b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f913c06b6d0 0x7f913c141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f913c13e460 0x7f913c0a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913bfff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f913c144910 0x7f913c1383a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40234/0 (socket says 172.21.15.19:40234) 2023-10-06T14:26:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913bfff640 1 -- 172.21.15.19:0/2286866872 learned_addr learned my addr 172.21.15.19:0/2286866872 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913bfff640 10 monclient(hunting): get_auth_request con 0x7f913c144910 auth_method 0 2023-10-06T14:26:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913bfff640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913bfff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913b7fe640 10 monclient(hunting): get_auth_request con 0x7f913c06b6d0 auth_method 0 2023-10-06T14:26:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913b7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913affd640 10 monclient(hunting): get_auth_request con 0x7f913c13e460 auth_method 0 2023-10-06T14:26:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913affd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913affd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.014+0000 7f913b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.015+0000 7f913affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.015+0000 7f913affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.015+0000 7f913affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.017+0000 7f913bfff640 10 monclient(hunting): handle_auth_done global_id 17129 payload 824 2023-10-06T14:26:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.017+0000 7f913bfff640 1 -- 172.21.15.19:0/2286866872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f913c13e460 msgr2=0x7f913c0a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.017+0000 7f913bfff640 1 --2- 172.21.15.19:0/2286866872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f913c13e460 0x7f913c0a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:27.026 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.017+0000 7f913bfff640 1 -- 172.21.15.19:0/2286866872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f913c06b6d0 msgr2=0x7f913c141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:27.026 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.017+0000 7f913bfff640 1 --2- 172.21.15.19:0/2286866872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f913c06b6d0 0x7f913c141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:27.026 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.017+0000 7f913bfff640 10 monclient: _finish_hunting 0 2023-10-06T14:26:27.026 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.017+0000 7f913bfff640 1 monclient: found mon.noname-c 2023-10-06T14:26:27.026 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.017+0000 7f913bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:27.026 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.017+0000 7f913bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T14:26:27.026 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.017+0000 7f913bfff640 1 -- 172.21.15.19:0/2286866872 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c0094d0 con 0x7f913c144910 2023-10-06T14:26:27.027 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.019+0000 7f913bfff640 1 --2- 172.21.15.19:0/2286866872 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f913c144910 0x7f913c1383a0 secure :-1 s=READY pgs=3120 cs=0 l=1 rev1=1 crypto rx=0x7f913400efc0 tx=0x7f913400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:27.028 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.019+0000 7f9123fff640 10 client.?.objecter ms_handle_connect 0x7f913c144910 2023-10-06T14:26:27.028 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.019+0000 7f9123fff640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:27.028 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.021+0000 7f9123fff640 1 -- 172.21.15.19:0/2286866872 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134014480 con 0x7f913c144910 2023-10-06T14:26:27.028 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.021+0000 7f9123fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:27.028 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.021+0000 7f9123fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T14:26:27.028 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.021+0000 7f9123fff640 10 monclient: dump: 2023-10-06T14:26:27.028 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:27.028 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:27.028 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:27.028 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:27.028 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:27.029 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:27.029 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:27.029 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:27.029 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:27.029 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:27.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.021+0000 7f9123fff640 10 monclient: _finish_auth 0 2023-10-06T14:26:27.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.021+0000 7f9123fff640 10 monclient: _check_auth_tickets 2023-10-06T14:26:27.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.021+0000 7f9123fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:27.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.021+0000 7f9141a7d640 5 monclient: authenticate success, global_id 17129 2023-10-06T14:26:27.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.021+0000 7f9141a7d640 10 monclient: _renew_subs 2023-10-06T14:26:27.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.021+0000 7f9141a7d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:27.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.021+0000 7f9141a7d640 1 -- 172.21.15.19:0/2286866872 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c1388e0 con 0x7f913c144910 2023-10-06T14:26:27.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.022+0000 7f9141a7d640 10 client.17129.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.022+0000 7f9141a7d640 10 monclient: _renew_subs 2023-10-06T14:26:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.022+0000 7f9141a7d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.022+0000 7f9141a7d640 1 -- 172.21.15.19:0/2286866872 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913c138d70 con 0x7f913c144910 2023-10-06T14:26:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.022+0000 7f9141a7d640 1 librados: init done 2023-10-06T14:26:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.022+0000 7f9141a7d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.022+0000 7f9141a7d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.022+0000 7f9141a7d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.022+0000 7f9141a7d640 1 -- 172.21.15.19:0/2286866872 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c0a8990 con 0x7f913c144910 2023-10-06T14:26:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.024+0000 7f9123fff640 1 -- 172.21.15.19:0/2286866872 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913400f040 con 0x7f913c144910 2023-10-06T14:26:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.024+0000 7f9123fff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.024+0000 7f9123fff640 1 -- 172.21.15.19:0/2286866872 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134014800 con 0x7f913c144910 2023-10-06T14:26:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.024+0000 7f9123fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.024+0000 7f9123fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.024+0000 7f9123fff640 10 monclient: dump: 2023-10-06T14:26:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:27.031 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.024+0000 7f9139ffb640 4 set_mon_vals no callback set 2023-10-06T14:26:27.047 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.045+0000 7f9123fff640 1 -- 172.21.15.19:0/2286866872 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f9134014960 con 0x7f913c144910 2023-10-06T14:26:27.047 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.045+0000 7f9123fff640 1 --2- 172.21.15.19:0/2286866872 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f912806c1c0 0x7f912806e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:27.047 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.045+0000 7f913b7fe640 1 --2- 172.21.15.19:0/2286866872 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f912806c1c0 0x7f912806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.045+0000 7f913b7fe640 10 monclient: get_auth_request con 0x7f912806c1c0 auth_method 0 2023-10-06T14:26:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.046+0000 7f913b7fe640 1 --2- 172.21.15.19:0/2286866872 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f912806c1c0 0x7f912806e670 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9130000f60 tx=0x7f9130009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.046+0000 7f9123fff640 10 client.17129.objecter ms_handle_connect 0x7f912806c1c0 2023-10-06T14:26:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.049+0000 7f9123fff640 1 -- 172.21.15.19:0/2286866872 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(969..969 src has 595..969) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f91340591b0 con 0x7f913c144910 2023-10-06T14:26:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.049+0000 7f9123fff640 10 client.17129.objecter ms_dispatch 0x7f913c000b90 osd_map(969..969 src has 595..969) v4 2023-10-06T14:26:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.049+0000 7f9123fff640 3 client.17129.objecter handle_osd_map got epochs [969,969] > 0 2023-10-06T14:26:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.049+0000 7f9123fff640 3 client.17129.objecter handle_osd_map decoding full epoch 969 2023-10-06T14:26:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.049+0000 7f9123fff640 20 client.17129.objecter dump_active .. 0 homeless 2023-10-06T14:26:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.313+0000 7f9123fff640 1 -- 172.21.15.19:0/2286866872 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f91340554f0 con 0x7f913c144910 2023-10-06T14:26:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.313+0000 7f9123fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:27.316 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.313+0000 7f9123fff640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:26:27.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.501+0000 7f9141a7d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-10-06T14:26:27.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.501+0000 7f9141a7d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T14:26:27.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.501+0000 7f9141a7d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:27.504 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.501+0000 7f9141a7d640 1 -- 172.21.15.19:0/2286866872 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f913c0af2c0 con 0x7f913c144910 2023-10-06T14:26:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.538+0000 7f9123fff640 1 -- 172.21.15.19:0/2286866872 <== mon.2 v2:172.21.15.19:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v969) v1 ==== 74+0+16997 (secure 0 0 0) 0x7f9134058b10 con 0x7f913c144910 2023-10-06T14:26:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.538+0000 7f9123fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-10-06T14:26:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.538+0000 7f9123fff640 10 monclient: _finish_command 2 = system:0 2023-10-06T14:26:27.541 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T14:26:27.542 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":969,"fsid":"c5a90f7c-81ec-43a3-ab47-af6ce5729ec5","created":"2023-10-06T13:52:19.516184+0000","modified":"2023-10-06T14:26:24.403839+0000","last_up_change":"2023-10-06T14:22:37.385938+0000","last_in_change":"2023-10-06T13:52:24.916896+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":106,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":259,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T13:53:07.784836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T13:53:14.364162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"ceph_test_rados_api_asio","create_time":"2023-10-06T13:54:14.565473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6cb75c5-86fb-48b9-895c-fb1d72bd4746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":963,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":86195},{"type":"v1","addr":"172.21.15.19:6818","nonce":86195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6821","nonce":86195},{"type":"v1","addr":"172.21.15.19:6823","nonce":86195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":86195},{"type":"v1","addr":"172.21.15.19:6831","nonce":86195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":86195},{"type":"v1","addr":"172.21.15.19:6827","nonce":86195}]},"public_addr":"172.21.15.19:6818/86195","cluster_addr":"172.21.15.19:6823/86195","heartbeat_back_addr":"172.21.15.19:6831/86195","heartbeat_front_addr":"172.21.15.19:6827/86195","state":["exists","up"]},{"osd":1,"uuid":"ba828cb4-a4f6-4fea-a02e-24d9cf597ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":963,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":86193},{"type":"v1","addr":"172.21.15.19:6819","nonce":86193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":86193},{"type":"v1","addr":"172.21.15.19:6822","nonce":86193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":86193},{"type":"v1","addr":"172.21.15.19:6830","nonce":86193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":86193},{"type":"v1","addr":"172.21.15.19:6826","nonce":86193}]},"public_addr":"172.21.15.19:6819/86193","cluster_addr":"172.21.15.19:6822/86193","heartbeat_back_addr":"172.21.15.19:6830/86193","heartbeat_front_addr":"172.21.15.19:6826/86193","state":["exists","up"]},{"osd":2,"uuid":"06bcfab5-d2f1-4447-ba44-143398707ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":963,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86194},{"type":"v1","addr":"172.21.15.19:6809","nonce":86194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":86194},{"type":"v1","addr":"172.21.15.19:6811","nonce":86194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":86194},{"type":"v1","addr":"172.21.15.19:6815","nonce":86194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":86194},{"type":"v1","addr":"172.21.15.19:6813","nonce":86194}]},"public_addr":"172.21.15.19:6809/86194","cluster_addr":"172.21.15.19:6811/86194","heartbeat_back_addr":"172.21.15.19:6815/86194","heartbeat_front_addr":"172.21.15.19:6813/86194","state":["exists","up"]},{"osd":3,"uuid":"7f72a3d8-8a3e-42af-bf65-e0d3e5199bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":832,"up_from":834,"up_thru":963,"down_at":831,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":86196},{"type":"v1","addr":"172.21.15.19:6801","nonce":86196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1086196},{"type":"v1","addr":"172.21.15.19:6833","nonce":1086196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86196},{"type":"v1","addr":"172.21.15.19:6807","nonce":86196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86196},{"type":"v1","addr":"172.21.15.19:6805","nonce":86196}]},"public_addr":"172.21.15.19:6801/86196","cluster_addr":"172.21.15.19:6833/1086196","heartbeat_back_addr":"172.21.15.19:6807/86196","heartbeat_front_addr":"172.21.15.19:6805/86196","state":["exists","up"]},{"osd":4,"uuid":"ad31d4df-e78c-4e32-bfd4-3fe6e70dd428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":428,"up_from":430,"up_thru":959,"down_at":411,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":86123},{"type":"v1","addr":"172.21.15.83:6827","nonce":86123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1086123},{"type":"v1","addr":"172.21.15.83:6835","nonce":1086123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":86123},{"type":"v1","addr":"172.21.15.83:6833","nonce":86123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":86123},{"type":"v1","addr":"172.21.15.83:6831","nonce":86123}]},"public_addr":"172.21.15.83:6827/86123","cluster_addr":"172.21.15.83:6835/1086123","heartbeat_back_addr":"172.21.15.83:6833/86123","heartbeat_front_addr":"172.21.15.83:6831/86123","state":["exists","up"]},{"osd":5,"uuid":"931b23f8-80c3-4595-8931-2568be479cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":657,"up_from":659,"up_thru":963,"down_at":651,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":86126},{"type":"v1","addr":"172.21.15.83:6819","nonce":86126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1086126},{"type":"v1","addr":"172.21.15.83:6829","nonce":1086126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":86126},{"type":"v1","addr":"172.21.15.83:6825","nonce":86126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":86126},{"type":"v1","addr":"172.21.15.83:6823","nonce":86126}]},"public_addr":"172.21.15.83:6819/86126","cluster_addr":"172.21.15.83:6829/1086126","heartbeat_back_addr":"172.21.15.83:6825/86126","heartbeat_front_addr":"172.21.15.83:6823/86126","state":["exists","up"]},{"osd":6,"uuid":"70dd2f95-93eb-448a-a65d-3a09bdea85a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":963,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":86124},{"type":"v1","addr":"172.21.15.83:6803","nonce":86124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":86124},{"type":"v1","addr":"172.21.15.83:6805","nonce":86124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":86124},{"type":"v1","addr":"172.21.15.83:6809","nonce":86124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":86124},{"type":"v1","addr":"172.21.15.83:6807","nonce":86124}]},"public_addr":"172.21.15.83:6803/86124","cluster_addr":"172.21.15.83:6805/86124","heartbeat_back_addr":"172.21.15.83:6809/86124","heartbeat_front_addr":"172.21.15.83:6807/86124","state":["exists","up"]},{"osd":7,"uuid":"9797eb84-1b2c-4280-ab13-bdb7b2007dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":963,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":86125},{"type":"v1","addr":"172.21.15.83:6811","nonce":86125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":86125},{"type":"v1","addr":"172.21.15.83:6813","nonce":86125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":86125},{"type":"v1","addr":"172.21.15.83:6817","nonce":86125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":86125},{"type":"v1","addr":"172.21.15.83:6815","nonce":86125}]},"public_addr":"172.21.15.83:6811/86125","cluster_addr":"172.21.15.83:6813/86125","heartbeat_back_addr":"172.21.15.83:6817/86125","heartbeat_front_addr":"172.21.15.83:6815/86125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:04.886714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:05.153337+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:04.794593+0000","dead_epoch":0},{"osd":3,"down_stamp":"2023-10-06T14:22:34.240341+0000","laggy_probability":0.30000001192092896,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:04.049373+0000","dead_epoch":831},{"osd":4,"down_stamp":"2023-10-06T14:04:50.087397+0000","laggy_probability":0.30000001192092896,"laggy_interval":6,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:03.567476+0000","dead_epoch":426},{"osd":5,"down_stamp":"2023-10-06T14:16:21.680911+0000","laggy_probability":0.30000001192092896,"laggy_interval":3,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:02.921086+0000","dead_epoch":656},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:02.431856+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T13:53:02.863561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":16,"snaps":[{"begin":9,"length":2}]},{"pool":55,"snaps":[{"begin":11,"length":2}]},{"pool":84,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":98,"snaps":[{"begin":2,"length":2},{"begin":6,"length":2}]},{"pool":107,"snaps":[{"begin":1,"length":1},{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":119,"snaps":[{"begin":2,"length":2},{"begin":6,"length":2},{"begin":10,"length":2}]},{"pool":176,"snaps":[{"begin":12,"length":1}]},{"pool":186,"snaps":[{"begin":14,"length":1}]},{"pool":193,"snaps":[{"begin":17,"length":1}]},{"pool":194,"snaps":[{"begin":19,"length":2}]},{"pool":197,"snaps":[{"begin":24,"length":1}]},{"pool":240,"snaps":[{"begin":2,"length":1}]},{"pool":241,"snaps":[{"begin":4,"length":1}]},{"pool":244,"snaps":[{"begin":6,"length":1}]},{"pool":248,"snaps":[{"begin":9,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T14:26:27.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.540+0000 7f9141a7d640 10 librados: watch_flush enter 2023-10-06T14:26:27.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.540+0000 7f9141a7d640 10 librados: watch_flush exit 2023-10-06T14:26:27.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.540+0000 7f9141a7d640 20 client.17129.objecter shutdown clearing up homeless session... 2023-10-06T14:26:27.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.540+0000 7f9141a7d640 10 client.17129.objecter successfully canceled tick 2023-10-06T14:26:27.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 1 -- 172.21.15.19:0/2286866872 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f912806c1c0 msgr2=0x7f912806e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:27.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 1 --2- 172.21.15.19:0/2286866872 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f912806c1c0 0x7f912806e670 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9130000f60 tx=0x7f9130009040 comp rx=0 tx=0).stop 2023-10-06T14:26:27.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 10 monclient: shutdown 2023-10-06T14:26:27.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:27.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 1 -- 172.21.15.19:0/2286866872 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f913c144910 msgr2=0x7f913c1383a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:27.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 1 --2- 172.21.15.19:0/2286866872 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f913c144910 0x7f913c1383a0 secure :-1 s=READY pgs=3120 cs=0 l=1 rev1=1 crypto rx=0x7f913400efc0 tx=0x7f913400c490 comp rx=0 tx=0).stop 2023-10-06T14:26:27.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 1 -- 172.21.15.19:0/2286866872 shutdown_connections 2023-10-06T14:26:27.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 1 --2- 172.21.15.19:0/2286866872 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f912806c1c0 0x7f912806e670 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:27.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 1 --2- 172.21.15.19:0/2286866872 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f913c144910 0x7f913c1383a0 unknown :-1 s=CLOSED pgs=3120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:27.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 1 --2- 172.21.15.19:0/2286866872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f913c13e460 0x7f913c0a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:27.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 1 --2- 172.21.15.19:0/2286866872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f913c06b6d0 0x7f913c141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:27.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 1 -- 172.21.15.19:0/2286866872 >> 172.21.15.19:0/2286866872 conn(0x7f913c0665b0 msgr2=0x7f913c0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:27.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 1 -- 172.21.15.19:0/2286866872 shutdown_connections 2023-10-06T14:26:27.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 1 -- 172.21.15.19:0/2286866872 wait complete. 2023-10-06T14:26:27.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:27.541+0000 7f9141a7d640 1 librados: shutdown 2023-10-06T14:26:28.452 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:31.574 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:28.457 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:31.580 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:28.478 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:31.600 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:28.500 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:31.597 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:28.503 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:31.600 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:28.515 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:31.612 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:28.534 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:31.656 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:28.537 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:31.660 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:28.552 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:31.674 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:28.556 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T14:26:28.556 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T14:26:28.677 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.674+0000 7fbb4e98b640 10 monclient: get_monmap_and_config 2023-10-06T14:26:28.677 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.674+0000 7fbb4e98b640 10 monclient: build_initial_monmap 2023-10-06T14:26:28.677 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.674+0000 7fbb4e98b640 10 monclient: monmap: 2023-10-06T14:26:28.677 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:28.677 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:28.677 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:28.675979+0000 2023-10-06T14:26:28.677 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:28.675979+0000 2023-10-06T14:26:28.677 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:28.677 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:28.677 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:28.678 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:28.678 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:28.678 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:28.678 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:28.678 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:28.679 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.675+0000 7fbb4e98b640 1 Processor -- start 2023-10-06T14:26:28.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.675+0000 7fbb4e98b640 1 -- start start 2023-10-06T14:26:28.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.675+0000 7fbb4e98b640 10 monclient: init 2023-10-06T14:26:28.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.676+0000 7fbb4e98b640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:28.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.676+0000 7fbb4e98b640 10 monclient: _add_conns ranks=[2,1,0,4,3] 2023-10-06T14:26:28.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.676+0000 7fbb4e98b640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbb480b3990 0x7fbb480af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:28.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.676+0000 7fbb4e98b640 10 monclient(hunting): picked mon.noname-b con 0x7fbb480b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:28.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.676+0000 7fbb4e98b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbb480b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T14:26:28.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.676+0000 7fbb4e98b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbb480b00c0 addr v1:172.21.15.19:6789/0 2023-10-06T14:26:28.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.676+0000 7fbb4e98b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb480aebe0 con 0x7fbb480b00c0 2023-10-06T14:26:28.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.676+0000 7fbb4e98b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb48141070 con 0x7fbb480b3d60 2023-10-06T14:26:28.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.676+0000 7fbb4e98b640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:28.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.676+0000 7fbb4e98b640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb48140dd0 con 0x7fbb480b3990 2023-10-06T14:26:28.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.676+0000 7fbb4e98b640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:28.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.676+0000 7fbb4e98b640 10 monclient(hunting): authenticate will time out at 3316.862762s 2023-10-06T14:26:28.686 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.684+0000 7fbb47fff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbb480b3990 0x7fbb480af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:28.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.685+0000 7fbb477fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbb480b3d60 0x7fbb48141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:42950/0 (socket says 172.21.15.19:42950) 2023-10-06T14:26:28.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.685+0000 7fbb477fe640 1 -- 172.21.15.19:0/396611323 learned_addr learned my addr 172.21.15.19:0/396611323 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T14:26:28.690 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.688+0000 7fbb47fff640 10 monclient(hunting): get_auth_request con 0x7fbb480b3990 auth_method 0 2023-10-06T14:26:28.690 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.688+0000 7fbb47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:28.690 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.688+0000 7fbb47fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:28.690 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.688+0000 7fbb47fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:28.692 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.690+0000 7fbb47fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:28.692 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.690+0000 7fbb47fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:28.692 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.690+0000 7fbb47fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.694+0000 7fbb47fff640 10 monclient(hunting): handle_auth_done global_id 17132 payload 274 2023-10-06T14:26:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.694+0000 7fbb47fff640 1 -- 172.21.15.19:0/396611323 >> v1:172.21.15.83:6789/0 conn(0x7fbb480b3d60 legacy=0x7fbb48141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.694+0000 7fbb47fff640 1 -- 172.21.15.19:0/396611323 >> v1:172.21.15.19:6789/0 conn(0x7fbb480b00c0 legacy=0x7fbb4814b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.694+0000 7fbb47fff640 10 monclient: _finish_hunting 0 2023-10-06T14:26:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.694+0000 7fbb47fff640 1 monclient: found mon.noname-b 2023-10-06T14:26:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.694+0000 7fbb47fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:28.697 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.694+0000 7fbb47fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T14:26:28.697 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.694+0000 7fbb47fff640 1 -- 172.21.15.19:0/396611323 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb48141c10 con 0x7fbb480b3990 2023-10-06T14:26:28.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.706+0000 7fbb47fff640 1 --2- 172.21.15.19:0/396611323 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbb480b3990 0x7fbb480af810 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 crypto rx=0x7fbb380097d0 tx=0x7fbb3802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=764f92a26b0b4c13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:28.732 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.729+0000 7fbb46ffd640 1 -- 172.21.15.19:0/396611323 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38031800 con 0x7fbb480b3990 2023-10-06T14:26:28.732 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb46ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:28.732 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb46ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:28.732 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb46ffd640 10 monclient: dump: 2023-10-06T14:26:28.732 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:28.732 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:28.732 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:28.732 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:28.733 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:28.733 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:28.733 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:28.733 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:28.733 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:28.733 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:28.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb46ffd640 10 monclient: _finish_auth 0 2023-10-06T14:26:28.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb46ffd640 10 monclient: _check_auth_tickets 2023-10-06T14:26:28.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb46ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:28.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb46ffd640 1 -- 172.21.15.19:0/396611323 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb38031960 con 0x7fbb480b3990 2023-10-06T14:26:28.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb46ffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:28.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb46ffd640 1 -- 172.21.15.19:0/396611323 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38031cf0 con 0x7fbb480b3990 2023-10-06T14:26:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb46ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb46ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb46ffd640 10 monclient: dump: 2023-10-06T14:26:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:28.735 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb4e98b640 5 monclient: authenticate success, global_id 17132 2023-10-06T14:26:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb4e98b640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb4e98b640 4 set_mon_vals no callback set 2023-10-06T14:26:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb4e98b640 10 monclient: shutdown 2023-10-06T14:26:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb4e98b640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb4e98b640 1 -- 172.21.15.19:0/396611323 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbb480b3990 msgr2=0x7fbb480af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.730+0000 7fbb4e98b640 1 --2- 172.21.15.19:0/396611323 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbb480b3990 0x7fbb480af810 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 crypto rx=0x7fbb380097d0 tx=0x7fbb3802b8a0 comp rx=0 tx=0).stop 2023-10-06T14:26:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.731+0000 7fbb4e98b640 1 -- 172.21.15.19:0/396611323 shutdown_connections 2023-10-06T14:26:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.731+0000 7fbb4e98b640 1 --2- 172.21.15.19:0/396611323 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbb480b3990 0x7fbb480af810 unknown :-1 s=CLOSED pgs=3121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.731+0000 7fbb4e98b640 1 -- 172.21.15.19:0/396611323 >> 172.21.15.19:0/396611323 conn(0x7fbb480b6e00 msgr2=0x7fbb480b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.731+0000 7fbb4e98b640 1 -- 172.21.15.19:0/396611323 shutdown_connections 2023-10-06T14:26:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.731+0000 7fbb4e98b640 1 -- 172.21.15.19:0/396611323 wait complete. 2023-10-06T14:26:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.732+0000 7fbb4e98b640 10 monclient: build_initial_monmap 2023-10-06T14:26:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.732+0000 7fbb4e98b640 10 monclient: monmap: 2023-10-06T14:26:28.739 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:28.739 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:28.739 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:28.739 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:28.739 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:28.739 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:28.739 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:28.739 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:28.739 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:28.739 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:28.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.732+0000 7fbb4e98b640 1 librados: starting msgr at 2023-10-06T14:26:28.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.732+0000 7fbb4e98b640 1 librados: starting objecter 2023-10-06T14:26:28.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.732+0000 7fbb4e98b640 1 Processor -- start 2023-10-06T14:26:28.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.732+0000 7fbb4e98b640 1 -- start start 2023-10-06T14:26:28.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.732+0000 7fbb4e98b640 1 librados: setting wanted keys 2023-10-06T14:26:28.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.732+0000 7fbb4e98b640 1 librados: calling monclient init 2023-10-06T14:26:28.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.732+0000 7fbb4e98b640 10 monclient: init 2023-10-06T14:26:28.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:28.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 10 monclient: _add_conns ranks=[2,1,0] 2023-10-06T14:26:28.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbb480b00c0 0x7fbb480abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:28.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 10 monclient(hunting): picked mon.noname-c con 0x7fbb480b00c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:28.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb480b3990 0x7fbb481d9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:28.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 10 monclient(hunting): picked mon.noname-b con 0x7fbb480b3990 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:28.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb480b3d60 0x7fbb48139fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:28.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 10 monclient(hunting): picked mon.noname-a con 0x7fbb480b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:28.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:28.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb480ac3e0 con 0x7fbb480b3d60 2023-10-06T14:26:28.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:28.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb481d9260 con 0x7fbb480b3990 2023-10-06T14:26:28.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:28.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb481d8f10 con 0x7fbb480b00c0 2023-10-06T14:26:28.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:28.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.733+0000 7fbb4e98b640 10 monclient(hunting): authenticate will time out at 3316.920065s 2023-10-06T14:26:28.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.739+0000 7fbb4cf28640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb480b3d60 0x7fbb48139fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:28.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.747+0000 7fbb4cf28640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb480b3d60 0x7fbb48139fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55286/0 (socket says 172.21.15.19:55286) 2023-10-06T14:26:28.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.747+0000 7fbb4cf28640 1 -- 172.21.15.19:0/3898887682 learned_addr learned my addr 172.21.15.19:0/3898887682 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:28.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.747+0000 7fbb4cf28640 10 monclient(hunting): get_auth_request con 0x7fbb480b3d60 auth_method 0 2023-10-06T14:26:28.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.747+0000 7fbb4cf28640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:28.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.747+0000 7fbb4cf28640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:28.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.747+0000 7fbb4cf28640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:28.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.747+0000 7fbb47fff640 1 --2- 172.21.15.19:0/3898887682 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbb480b00c0 0x7fbb480abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:28.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.750+0000 7fbb4cf28640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:28.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.750+0000 7fbb4cf28640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:28.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.750+0000 7fbb4cf28640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:28.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.751+0000 7fbb47fff640 10 monclient(hunting): get_auth_request con 0x7fbb480b00c0 auth_method 0 2023-10-06T14:26:28.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.751+0000 7fbb47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:28.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.751+0000 7fbb47fff640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:28.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.751+0000 7fbb47fff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:28.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.752+0000 7fbb47fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:28.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.752+0000 7fbb47fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:28.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.752+0000 7fbb47fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.757+0000 7fbb4cf28640 10 monclient(hunting): handle_auth_done global_id 15621 payload 824 2023-10-06T14:26:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.757+0000 7fbb4cf28640 1 -- 172.21.15.19:0/3898887682 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbb480b00c0 msgr2=0x7fbb480abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.757+0000 7fbb4cf28640 1 --2- 172.21.15.19:0/3898887682 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fbb480b00c0 0x7fbb480abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.757+0000 7fbb4cf28640 1 -- 172.21.15.19:0/3898887682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb480b3990 msgr2=0x7fbb481d9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.757+0000 7fbb4cf28640 1 --2- 172.21.15.19:0/3898887682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb480b3990 0x7fbb481d9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.757+0000 7fbb4cf28640 10 monclient: _finish_hunting 0 2023-10-06T14:26:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.757+0000 7fbb4cf28640 1 monclient: found mon.noname-a 2023-10-06T14:26:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.757+0000 7fbb4cf28640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.757+0000 7fbb4cf28640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T14:26:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.757+0000 7fbb4cf28640 1 -- 172.21.15.19:0/3898887682 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb38009400 con 0x7fbb480b3d60 2023-10-06T14:26:28.767 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.766+0000 7fbb4cf28640 1 --2- 172.21.15.19:0/3898887682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb480b3d60 0x7fbb48139fb0 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000c7a0 tx=0x7fbb4000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:28.768 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.766+0000 7fbb44ff9640 10 client.?.objecter ms_handle_connect 0x7fbb480b3d60 2023-10-06T14:26:28.768 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.766+0000 7fbb44ff9640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb44ff9640 1 -- 172.21.15.19:0/3898887682 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40003b70 con 0x7fbb480b3d60 2023-10-06T14:26:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb44ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb44ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T14:26:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb44ff9640 10 monclient: dump: 2023-10-06T14:26:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:28.823 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb44ff9640 10 monclient: _finish_auth 0 2023-10-06T14:26:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb44ff9640 10 monclient: _check_auth_tickets 2023-10-06T14:26:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb44ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb4e98b640 5 monclient: authenticate success, global_id 15621 2023-10-06T14:26:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb44ff9640 1 -- 172.21.15.19:0/3898887682 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb40003cf0 con 0x7fbb480b3d60 2023-10-06T14:26:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb4e98b640 10 monclient: _renew_subs 2023-10-06T14:26:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb4e98b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T14:26:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb4e98b640 1 -- 172.21.15.19:0/3898887682 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb481d9cc0 con 0x7fbb480b3d60 2023-10-06T14:26:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb44ff9640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb44ff9640 1 -- 172.21.15.19:0/3898887682 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000aa90 con 0x7fbb480b3d60 2023-10-06T14:26:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb44ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb44ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T14:26:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb44ff9640 10 monclient: dump: 2023-10-06T14:26:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:28.825 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb467fc640 4 set_mon_vals no callback set 2023-10-06T14:26:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb4e98b640 10 client.15621.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb4e98b640 10 monclient: _renew_subs 2023-10-06T14:26:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb4e98b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T14:26:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb4e98b640 1 -- 172.21.15.19:0/3898887682 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4813a6d0 con 0x7fbb480b3d60 2023-10-06T14:26:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.819+0000 7fbb4e98b640 1 librados: init done 2023-10-06T14:26:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.820+0000 7fbb4e98b640 10 client.15621.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.820+0000 7fbb4e98b640 15 client.15621.objecter _session_command_op_assign -1 1 2023-10-06T14:26:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.820+0000 7fbb4e98b640 20 client.15621.objecter _get_session osd=-1 returning homeless 2023-10-06T14:26:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.820+0000 7fbb4e98b640 10 client.15621.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.820+0000 7fbb4e98b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T14:26:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.820+0000 7fbb4e98b640 1 -- 172.21.15.19:0/3898887682 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb480af3f0 con 0x7fbb480b3d60 2023-10-06T14:26:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.848+0000 7fbb44ff9640 1 -- 172.21.15.19:0/3898887682 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fbb4000abf0 con 0x7fbb480b3d60 2023-10-06T14:26:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.848+0000 7fbb44ff9640 1 --2- 172.21.15.19:0/3898887682 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbb3406c210 0x7fbb3406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:28.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.849+0000 7fbb47fff640 1 --2- 172.21.15.19:0/3898887682 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbb3406c210 0x7fbb3406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:28.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.849+0000 7fbb47fff640 10 monclient: get_auth_request con 0x7fbb3406c210 auth_method 0 2023-10-06T14:26:28.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.849+0000 7fbb47fff640 1 --2- 172.21.15.19:0/3898887682 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbb3406c210 0x7fbb3406e6c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbb38036000 tx=0x7fbb3803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:28.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.849+0000 7fbb44ff9640 10 client.15621.objecter ms_handle_connect 0x7fbb3406c210 2023-10-06T14:26:28.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 1 -- 172.21.15.19:0/3898887682 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(970..970 src has 595..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fbb40058da0 con 0x7fbb480b3d60 2023-10-06T14:26:28.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 10 client.15621.objecter ms_dispatch 0x7fbb48000b90 osd_map(970..970 src has 595..970) v4 2023-10-06T14:26:28.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 3 client.15621.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:28.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 3 client.15621.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:28.864 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 10 client.15621.objecter checking command 1 2023-10-06T14:26:28.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 1 --2- 172.21.15.19:0/3898887682 >> [v2:172.21.15.19:6816/86195,v1:172.21.15.19:6818/86195] conn(0x7fbb34073a30 0x7fbb34075ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:28.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 20 client.15621.objecter _get_session s=0x7fbb34073360 osd=0 3 2023-10-06T14:26:28.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 20 client.15621.objecter put_session s=0x7fbb34073360 osd=0 3 2023-10-06T14:26:28.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 15 client.15621.objecter _session_command_op_remove -1 1 2023-10-06T14:26:28.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 20 client.15621.objecter _get_session s=0x7fbb34073360 osd=0 3 2023-10-06T14:26:28.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 20 client.15621.objecter get_session s=0x7fbb34073360 osd=0 3 2023-10-06T14:26:28.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 15 client.15621.objecter _session_command_op_assign 0 1 2023-10-06T14:26:28.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 20 client.15621.objecter put_session s=0x7fbb34073360 osd=0 4 2023-10-06T14:26:28.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 10 client.15621.objecter _send_command 1 2023-10-06T14:26:28.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 1 -- 172.21.15.19:0/3898887682 --> [v2:172.21.15.19:6816/86195,v1:172.21.15.19:6818/86195] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb34076590 con 0x7fbb34073a30 2023-10-06T14:26:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 20 client.15621.objecter dump_active .. 0 homeless 2023-10-06T14:26:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 1 -- 172.21.15.19:0/3898887682 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=970) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb4000aea0 con 0x7fbb480b3d60 2023-10-06T14:26:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.861+0000 7fbb44ff9640 10 monclient: handle_get_version_reply finishing 1 version 970 2023-10-06T14:26:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.862+0000 7fbb477fe640 1 --2- 172.21.15.19:0/3898887682 >> [v2:172.21.15.19:6816/86195,v1:172.21.15.19:6818/86195] conn(0x7fbb34073a30 0x7fbb34075ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.862+0000 7fbb477fe640 10 monclient: get_auth_request con 0x7fbb34073a30 auth_method 0 2023-10-06T14:26:28.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.866+0000 7fbb477fe640 1 --2- 172.21.15.19:0/3898887682 >> [v2:172.21.15.19:6816/86195,v1:172.21.15.19:6818/86195] conn(0x7fbb34073a30 0x7fbb34075ec0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:28.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.866+0000 7fbb44ff9640 10 client.15621.objecter ms_handle_connect 0x7fbb34073a30 2023-10-06T14:26:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.890+0000 7fbb44ff9640 1 -- 172.21.15.19:0/3898887682 <== osd.0 v2:172.21.15.19:6816/86195 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbb34076590 con 0x7fbb34073a30 2023-10-06T14:26:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.890+0000 7fbb44ff9640 10 client.15621.objecter ms_dispatch 0x7fbb48000b90 command_reply(tid 1: 0 ) v1 2023-10-06T14:26:28.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.890+0000 7fbb44ff9640 10 client.15621.objecter _finish_command 1 = system:0 2023-10-06T14:26:28.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.890+0000 7fbb44ff9640 20 client.15621.objecter put_session s=0x7fbb34073360 osd=0 4 2023-10-06T14:26:28.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.890+0000 7fbb44ff9640 15 client.15621.objecter _session_command_op_remove 0 1 2023-10-06T14:26:28.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.917+0000 7fbb4e98b640 10 client.15621.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-10-06T14:26:28.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.917+0000 7fbb4e98b640 15 client.15621.objecter _session_command_op_assign -1 2 2023-10-06T14:26:28.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.917+0000 7fbb4e98b640 20 client.15621.objecter _get_session s=0x7fbb34073360 osd=0 3 2023-10-06T14:26:28.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.917+0000 7fbb4e98b640 20 client.15621.objecter put_session s=0x7fbb34073360 osd=0 3 2023-10-06T14:26:28.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.917+0000 7fbb4e98b640 20 client.15621.objecter _get_session s=0x7fbb34073360 osd=0 3 2023-10-06T14:26:28.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.917+0000 7fbb4e98b640 15 client.15621.objecter _session_command_op_remove -1 2 2023-10-06T14:26:28.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.917+0000 7fbb4e98b640 20 client.15621.objecter get_session s=0x7fbb34073360 osd=0 3 2023-10-06T14:26:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.917+0000 7fbb4e98b640 15 client.15621.objecter _session_command_op_assign 0 2 2023-10-06T14:26:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.917+0000 7fbb4e98b640 20 client.15621.objecter put_session s=0x7fbb34073360 osd=0 4 2023-10-06T14:26:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.917+0000 7fbb4e98b640 10 client.15621.objecter _send_command 2 2023-10-06T14:26:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.917+0000 7fbb4e98b640 1 -- 172.21.15.19:0/3898887682 --> [v2:172.21.15.19:6816/86195,v1:172.21.15.19:6818/86195] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbb480af110 con 0x7fbb34073a30 2023-10-06T14:26:28.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb44ff9640 1 -- 172.21.15.19:0/3898887682 <== osd.0 v2:172.21.15.19:6816/86195 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbb3c001210 con 0x7fbb34073a30 2023-10-06T14:26:28.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb44ff9640 10 client.15621.objecter ms_dispatch 0x7fbb48000b90 command_reply(tid 2: 0 ) v1 2023-10-06T14:26:28.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb44ff9640 10 client.15621.objecter _finish_command 2 = system:0 2023-10-06T14:26:28.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb44ff9640 20 client.15621.objecter put_session s=0x7fbb34073360 osd=0 4 2023-10-06T14:26:28.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb44ff9640 15 client.15621.objecter _session_command_op_remove 0 2 2023-10-06T14:26:28.936 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2023-10-06T14:26:28.936 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T14:26:28.936 INFO:teuthology.orchestra.run.smithi019.stdout:} 2023-10-06T14:26:28.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4e98b640 10 librados: watch_flush enter 2023-10-06T14:26:28.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4e98b640 10 librados: watch_flush exit 2023-10-06T14:26:28.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4e98b640 10 client.15621.objecter close_session for osd.0 2023-10-06T14:26:28.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4e98b640 1 -- 172.21.15.19:0/3898887682 >> [v2:172.21.15.19:6816/86195,v1:172.21.15.19:6818/86195] conn(0x7fbb34073a30 msgr2=0x7fbb34075ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:28.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4e98b640 1 --2- 172.21.15.19:0/3898887682 >> [v2:172.21.15.19:6816/86195,v1:172.21.15.19:6818/86195] conn(0x7fbb34073a30 0x7fbb34075ec0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:28.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4e98b640 20 client.15621.objecter put_session s=0x7fbb34073360 osd=0 1 2023-10-06T14:26:28.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4e98b640 20 client.15621.objecter shutdown clearing up homeless session... 2023-10-06T14:26:28.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4e98b640 10 client.15621.objecter successfully canceled tick 2023-10-06T14:26:28.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4e98b640 1 -- 172.21.15.19:0/3898887682 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbb3406c210 msgr2=0x7fbb3406e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:28.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4e98b640 1 --2- 172.21.15.19:0/3898887682 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fbb3406c210 0x7fbb3406e6c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbb38036000 tx=0x7fbb3803c040 comp rx=0 tx=0).stop 2023-10-06T14:26:28.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4e98b640 10 monclient: shutdown 2023-10-06T14:26:28.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4e98b640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:28.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4e98b640 1 -- 172.21.15.19:0/3898887682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb480b3d60 msgr2=0x7fbb48139fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:28.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4e98b640 1 --2- 172.21.15.19:0/3898887682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb480b3d60 0x7fbb48139fb0 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000c7a0 tx=0x7fbb4000cc60 comp rx=0 tx=0).stop 2023-10-06T14:26:28.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.933+0000 7fbb4cf28640 1 -- 172.21.15.19:0/3898887682 reap_dead start 2023-10-06T14:26:28.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.934+0000 7fbb4e98b640 1 -- 172.21.15.19:0/3898887682 shutdown_connections 2023-10-06T14:26:28.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.934+0000 7fbb4e98b640 1 -- 172.21.15.19:0/3898887682 >> 172.21.15.19:0/3898887682 conn(0x7fbb480b6e00 msgr2=0x7fbb480b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:28.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.934+0000 7fbb4e98b640 1 -- 172.21.15.19:0/3898887682 shutdown_connections 2023-10-06T14:26:28.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.934+0000 7fbb4e98b640 1 -- 172.21.15.19:0/3898887682 wait complete. 2023-10-06T14:26:28.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:28.934+0000 7fbb4e98b640 1 librados: shutdown 2023-10-06T14:26:28.950 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T14:26:29.059 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.057+0000 7f0a72651640 10 monclient: get_monmap_and_config 2023-10-06T14:26:29.086 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.057+0000 7f0a72651640 10 monclient: build_initial_monmap 2023-10-06T14:26:29.086 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.057+0000 7f0a72651640 10 monclient: monmap: 2023-10-06T14:26:29.086 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:29.086 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.086 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:29.058242+0000 2023-10-06T14:26:29.086 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:29.058242+0000 2023-10-06T14:26:29.086 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:29.086 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.086 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:29.087 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:29.087 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:29.087 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:29.087 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:29.087 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.089 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 1 Processor -- start 2023-10-06T14:26:29.089 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 1 -- start start 2023-10-06T14:26:29.089 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 10 monclient: init 2023-10-06T14:26:29.089 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:29.089 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 10 monclient: _add_conns ranks=[4,3,0,1,2] 2023-10-06T14:26:29.089 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a6c0b3790 0x7f0a6c0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 10 monclient(hunting): picked mon.noname-c con 0x7f0a6c0b3790 addr v2:172.21.15.83:3300/0 2023-10-06T14:26:29.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0a6c0b1f50 0x7f0a6c144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 10 monclient(hunting): picked mon.noname-a con 0x7f0a6c0b1f50 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:29.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0a6c0b23b0 addr v1:172.21.15.19:6789/0 2023-10-06T14:26:29.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6c0b3e00 con 0x7f0a6c0b23b0 2023-10-06T14:26:29.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:29.090 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a6c0b3b60 con 0x7f0a6c0b1f50 2023-10-06T14:26:29.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:29.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a6c0ae740 con 0x7f0a6c0b3790 2023-10-06T14:26:29.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:29.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.058+0000 7f0a72651640 10 monclient(hunting): authenticate will time out at 3317.245214s 2023-10-06T14:26:29.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.063+0000 7f0a6b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0a6c0b1f50 0x7f0a6c144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.066+0000 7f0a6b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0a6c0b1f50 0x7f0a6c144540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55292/0 (socket says 172.21.15.19:55292) 2023-10-06T14:26:29.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.066+0000 7f0a6b7fe640 1 -- 172.21.15.19:0/2919366602 learned_addr learned my addr 172.21.15.19:0/2919366602 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:29.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.066+0000 7f0a6b7fe640 10 monclient(hunting): get_auth_request con 0x7f0a6c0b1f50 auth_method 0 2023-10-06T14:26:29.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.066+0000 7f0a6b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:29.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.066+0000 7f0a6b7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:29.091 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.066+0000 7f0a6b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:29.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.070+0000 7f0a6bfff640 1 --2- 172.21.15.19:0/2919366602 >> v2:172.21.15.83:3300/0 conn(0x7f0a6c0b3790 0x7f0a6c0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.073+0000 7f0a6bfff640 10 monclient(hunting): get_auth_request con 0x7f0a6c0b3790 auth_method 0 2023-10-06T14:26:29.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.073+0000 7f0a6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:29.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.073+0000 7f0a6bfff640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:29.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.073+0000 7f0a6bfff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:29.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.073+0000 7f0a6b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:29.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.073+0000 7f0a6b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:29.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.073+0000 7f0a6b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:29.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.073+0000 7f0a6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:29.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.073+0000 7f0a6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:29.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.073+0000 7f0a6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:29.092 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.076+0000 7f0a6b7fe640 10 monclient(hunting): handle_auth_done global_id 15624 payload 274 2023-10-06T14:26:29.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.076+0000 7f0a6b7fe640 1 -- 172.21.15.19:0/2919366602 >> v2:172.21.15.83:3300/0 conn(0x7f0a6c0b3790 msgr2=0x7f0a6c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:29.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.076+0000 7f0a6b7fe640 1 --2- 172.21.15.19:0/2919366602 >> v2:172.21.15.83:3300/0 conn(0x7f0a6c0b3790 0x7f0a6c0b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.076+0000 7f0a6b7fe640 1 -- 172.21.15.19:0/2919366602 >> v1:172.21.15.19:6789/0 conn(0x7f0a6c0b23b0 legacy=0x7f0a6c144a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.076+0000 7f0a6b7fe640 10 monclient: _finish_hunting 0 2023-10-06T14:26:29.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.076+0000 7f0a6b7fe640 1 monclient: found mon.noname-a 2023-10-06T14:26:29.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.076+0000 7f0a6b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:29.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.076+0000 7f0a6b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T14:26:29.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.076+0000 7f0a6b7fe640 1 -- 172.21.15.19:0/2919366602 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c0aea20 con 0x7f0a6c0b1f50 2023-10-06T14:26:29.093 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.086+0000 7f0a6b7fe640 1 --2- 172.21.15.19:0/2919366602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6c0b1f50 0x7f0a6c144540 secure :-1 s=READY pgs=3184 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c073b50 tx=0x7f0a6002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206c5d955336f0a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:29.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.091+0000 7f0a6affd640 1 -- 172.21.15.19:0/2919366602 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60031870 con 0x7f0a6c0b1f50 2023-10-06T14:26:29.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.091+0000 7f0a6affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:29.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.091+0000 7f0a6affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T14:26:29.094 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.091+0000 7f0a6affd640 10 monclient: dump: 2023-10-06T14:26:29.094 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:29.094 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.094 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.094 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.094 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:29.094 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.095 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:29.095 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:29.095 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:29.095 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.095 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.091+0000 7f0a6affd640 10 monclient: _finish_auth 0 2023-10-06T14:26:29.095 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.091+0000 7f0a6affd640 10 monclient: _check_auth_tickets 2023-10-06T14:26:29.095 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.091+0000 7f0a6affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:29.095 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.091+0000 7f0a72651640 5 monclient: authenticate success, global_id 15624 2023-10-06T14:26:29.095 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.091+0000 7f0a72651640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:29.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.137+0000 7f0a6affd640 1 -- 172.21.15.19:0/2919366602 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a600319d0 con 0x7f0a6c0b1f50 2023-10-06T14:26:29.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.137+0000 7f0a6affd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:29.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.137+0000 7f0a6affd640 1 -- 172.21.15.19:0/2919366602 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60031d60 con 0x7f0a6c0b1f50 2023-10-06T14:26:29.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.137+0000 7f0a6affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:29.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.137+0000 7f0a6affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T14:26:29.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.137+0000 7f0a6affd640 10 monclient: dump: 2023-10-06T14:26:29.139 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:29.139 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.140 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.140 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.140 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:29.140 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.140 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:29.140 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:29.140 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:29.140 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.140 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.137+0000 7f0a72651640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:29.140 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.137+0000 7f0a72651640 4 set_mon_vals no callback set 2023-10-06T14:26:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.138+0000 7f0a72651640 10 monclient: shutdown 2023-10-06T14:26:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.138+0000 7f0a72651640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.138+0000 7f0a72651640 1 -- 172.21.15.19:0/2919366602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6c0b1f50 msgr2=0x7f0a6c144540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.138+0000 7f0a72651640 1 --2- 172.21.15.19:0/2919366602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6c0b1f50 0x7f0a6c144540 secure :-1 s=READY pgs=3184 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c073b50 tx=0x7f0a6002b910 comp rx=0 tx=0).stop 2023-10-06T14:26:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.138+0000 7f0a72651640 1 -- 172.21.15.19:0/2919366602 shutdown_connections 2023-10-06T14:26:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.138+0000 7f0a72651640 1 --2- 172.21.15.19:0/2919366602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6c0b1f50 0x7f0a6c144540 unknown :-1 s=CLOSED pgs=3184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.138+0000 7f0a72651640 1 --2- 172.21.15.19:0/2919366602 >> v2:172.21.15.83:3300/0 conn(0x7f0a6c0b3790 0x7f0a6c0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.138+0000 7f0a72651640 1 -- 172.21.15.19:0/2919366602 >> 172.21.15.19:0/2919366602 conn(0x7f0a6c06a2d0 msgr2=0x7f0a6c06a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.138+0000 7f0a72651640 1 -- 172.21.15.19:0/2919366602 shutdown_connections 2023-10-06T14:26:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.138+0000 7f0a72651640 1 -- 172.21.15.19:0/2919366602 wait complete. 2023-10-06T14:26:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.138+0000 7f0a72651640 10 monclient: build_initial_monmap 2023-10-06T14:26:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.138+0000 7f0a72651640 10 monclient: monmap: 2023-10-06T14:26:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:29.145 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.139+0000 7f0a72651640 1 librados: starting msgr at 2023-10-06T14:26:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.139+0000 7f0a72651640 1 librados: starting objecter 2023-10-06T14:26:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.139+0000 7f0a72651640 1 Processor -- start 2023-10-06T14:26:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.139+0000 7f0a72651640 1 -- start start 2023-10-06T14:26:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.139+0000 7f0a72651640 1 librados: setting wanted keys 2023-10-06T14:26:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.139+0000 7f0a72651640 1 librados: calling monclient init 2023-10-06T14:26:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.139+0000 7f0a72651640 10 monclient: init 2023-10-06T14:26:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T14:26:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6c0b1f50 0x7f0a6c0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 10 monclient(hunting): picked mon.noname-a con 0x7f0a6c0b1f50 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a6c0b23b0 0x7f0a6c1d9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 10 monclient(hunting): picked mon.noname-b con 0x7f0a6c0b23b0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0a6c0b3790 0x7f0a6c1d5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 10 monclient(hunting): picked mon.noname-c con 0x7f0a6c0b3790 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c0ad500 con 0x7f0a6c0b1f50 2023-10-06T14:26:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1d9540 con 0x7f0a6c0b23b0 2023-10-06T14:26:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1d5fd0 con 0x7f0a6c0b3790 2023-10-06T14:26:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a72651640 10 monclient(hunting): authenticate will time out at 3317.326442s 2023-10-06T14:26:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.140+0000 7f0a6b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a6c0b23b0 0x7f0a6c1d9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.141+0000 7f0a6b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a6c0b23b0 0x7f0a6c1d9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:60508/0 (socket says 172.21.15.19:60508) 2023-10-06T14:26:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.141+0000 7f0a6b7fe640 1 -- 172.21.15.19:0/1380431200 learned_addr learned my addr 172.21.15.19:0/1380431200 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.141+0000 7f0a6b7fe640 10 monclient(hunting): get_auth_request con 0x7f0a6c0b23b0 auth_method 0 2023-10-06T14:26:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.141+0000 7f0a6b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.141+0000 7f0a6b7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.141+0000 7f0a6b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.141+0000 7f0a6b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.141+0000 7f0a6b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.141+0000 7f0a6b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.142+0000 7f0a70bee640 1 --2- 172.21.15.19:0/1380431200 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0a6c0b3790 0x7f0a6c1d5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.143+0000 7f0a70bee640 10 monclient(hunting): get_auth_request con 0x7f0a6c0b3790 auth_method 0 2023-10-06T14:26:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.143+0000 7f0a70bee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.143+0000 7f0a70bee640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.143+0000 7f0a70bee640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.143+0000 7f0a6bfff640 1 --2- 172.21.15.19:0/1380431200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6c0b1f50 0x7f0a6c0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.144+0000 7f0a6b7fe640 10 monclient(hunting): handle_auth_done global_id 20740 payload 824 2023-10-06T14:26:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.144+0000 7f0a6b7fe640 1 -- 172.21.15.19:0/1380431200 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0a6c0b3790 msgr2=0x7f0a6c1d5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.144+0000 7f0a6b7fe640 1 --2- 172.21.15.19:0/1380431200 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0a6c0b3790 0x7f0a6c1d5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.144+0000 7f0a6b7fe640 1 -- 172.21.15.19:0/1380431200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6c0b1f50 msgr2=0x7f0a6c0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.144+0000 7f0a6b7fe640 1 --2- 172.21.15.19:0/1380431200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6c0b1f50 0x7f0a6c0ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.144+0000 7f0a6b7fe640 10 monclient: _finish_hunting 0 2023-10-06T14:26:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.144+0000 7f0a6b7fe640 1 monclient: found mon.noname-b 2023-10-06T14:26:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.144+0000 7f0a6b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.144+0000 7f0a6b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T14:26:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.144+0000 7f0a6b7fe640 1 -- 172.21.15.19:0/1380431200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a600094d0 con 0x7f0a6c0b23b0 2023-10-06T14:26:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.146+0000 7f0a6b7fe640 1 --2- 172.21.15.19:0/1380431200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a6c0b23b0 0x7f0a6c1d9a60 secure :-1 s=READY pgs=5107 cs=0 l=1 rev1=1 crypto rx=0x7f0a6002b950 tx=0x7f0a600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.146+0000 7f0a68ff9640 10 client.?.objecter ms_handle_connect 0x7f0a6c0b23b0 2023-10-06T14:26:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.146+0000 7f0a68ff9640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a68ff9640 1 -- 172.21.15.19:0/1380431200 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6003abe0 con 0x7f0a6c0b23b0 2023-10-06T14:26:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a68ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a68ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a68ff9640 10 monclient: dump: 2023-10-06T14:26:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:29.153 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.153 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.153 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.153 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:29.153 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.153 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:29.153 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:29.153 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:29.153 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.153 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a68ff9640 10 monclient: _finish_auth 0 2023-10-06T14:26:29.153 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a68ff9640 10 monclient: _check_auth_tickets 2023-10-06T14:26:29.153 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a68ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a68ff9640 1 -- 172.21.15.19:0/1380431200 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6003ad60 con 0x7f0a6c0b23b0 2023-10-06T14:26:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a68ff9640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a68ff9640 1 -- 172.21.15.19:0/1380431200 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60044630 con 0x7f0a6c0b23b0 2023-10-06T14:26:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a72651640 5 monclient: authenticate success, global_id 20740 2023-10-06T14:26:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a72651640 10 monclient: _renew_subs 2023-10-06T14:26:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a72651640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a72651640 1 -- 172.21.15.19:0/1380431200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6c0a9890 con 0x7f0a6c0b23b0 2023-10-06T14:26:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a6a7fc640 4 set_mon_vals no callback set 2023-10-06T14:26:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a68ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a68ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a68ff9640 10 monclient: dump: 2023-10-06T14:26:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:29.155 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a72651640 10 client.20740.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a72651640 10 monclient: _renew_subs 2023-10-06T14:26:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a72651640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:29.156 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a72651640 1 -- 172.21.15.19:0/1380431200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6c1d8f90 con 0x7f0a6c0b23b0 2023-10-06T14:26:29.156 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.150+0000 7f0a72651640 1 librados: init done 2023-10-06T14:26:29.156 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.151+0000 7f0a72651640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:29.156 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.151+0000 7f0a72651640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:29.156 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.151+0000 7f0a72651640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:29.156 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.151+0000 7f0a72651640 1 -- 172.21.15.19:0/1380431200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6c1d6190 con 0x7f0a6c0b23b0 2023-10-06T14:26:29.389 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.386+0000 7f0a68ff9640 1 -- 172.21.15.19:0/1380431200 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0a6004d450 con 0x7f0a6c0b23b0 2023-10-06T14:26:29.389 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.387+0000 7f0a68ff9640 1 --2- 172.21.15.19:0/1380431200 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0a5806c1c0 0x7f0a5806e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.389 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.387+0000 7f0a6bfff640 1 --2- 172.21.15.19:0/1380431200 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0a5806c1c0 0x7f0a5806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.387+0000 7f0a6bfff640 10 monclient: get_auth_request con 0x7f0a5806c1c0 auth_method 0 2023-10-06T14:26:29.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.388+0000 7f0a68ff9640 10 client.20740.objecter ms_handle_connect 0x7f0a5806c1c0 2023-10-06T14:26:29.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.388+0000 7f0a6bfff640 1 --2- 172.21.15.19:0/1380431200 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0a5806c1c0 0x7f0a5806e670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009960 tx=0x7f0a5c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:29.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.389+0000 7f0a68ff9640 1 -- 172.21.15.19:0/1380431200 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(970..970 src has 595..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f0a6002d040 con 0x7f0a6c0b23b0 2023-10-06T14:26:29.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.389+0000 7f0a68ff9640 10 client.20740.objecter ms_dispatch 0x7f0a6c000b90 osd_map(970..970 src has 595..970) v4 2023-10-06T14:26:29.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.389+0000 7f0a68ff9640 3 client.20740.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:29.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.389+0000 7f0a68ff9640 3 client.20740.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:29.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.389+0000 7f0a68ff9640 20 client.20740.objecter dump_active .. 0 homeless 2023-10-06T14:26:29.400 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.398+0000 7f0a68ff9640 1 -- 172.21.15.19:0/1380431200 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f0a60076c90 con 0x7f0a6c0b23b0 2023-10-06T14:26:29.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.398+0000 7f0a68ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:29.401 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.398+0000 7f0a68ff9640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:26:29.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.587+0000 7f0a72651640 1 -- 172.21.15.19:0/1380431200 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a6c0aefe0 con 0x7f0a5806c1c0 2023-10-06T14:26:29.590 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.588+0000 7f0a68ff9640 1 -- 172.21.15.19:0/1380431200 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0a5c010870 con 0x7f0a5806c1c0 2023-10-06T14:26:29.590 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T14:26:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 10 librados: watch_flush enter 2023-10-06T14:26:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 10 librados: watch_flush exit 2023-10-06T14:26:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 20 client.20740.objecter shutdown clearing up homeless session... 2023-10-06T14:26:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 10 client.20740.objecter successfully canceled tick 2023-10-06T14:26:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 1 -- 172.21.15.19:0/1380431200 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0a5806c1c0 msgr2=0x7f0a5806e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 1 --2- 172.21.15.19:0/1380431200 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0a5806c1c0 0x7f0a5806e670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009960 tx=0x7f0a5c008040 comp rx=0 tx=0).stop 2023-10-06T14:26:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 10 monclient: shutdown 2023-10-06T14:26:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:29.594 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 1 -- 172.21.15.19:0/1380431200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a6c0b23b0 msgr2=0x7f0a6c1d9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.594 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 1 --2- 172.21.15.19:0/1380431200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a6c0b23b0 0x7f0a6c1d9a60 secure :-1 s=READY pgs=5107 cs=0 l=1 rev1=1 crypto rx=0x7f0a6002b950 tx=0x7f0a600046c0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.594 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 1 -- 172.21.15.19:0/1380431200 shutdown_connections 2023-10-06T14:26:29.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 1 --2- 172.21.15.19:0/1380431200 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f0a5806c1c0 0x7f0a5806e670 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 1 --2- 172.21.15.19:0/1380431200 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f0a6c0b3790 0x7f0a6c1d5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 1 --2- 172.21.15.19:0/1380431200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a6c0b23b0 0x7f0a6c1d9a60 unknown :-1 s=CLOSED pgs=5107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 1 --2- 172.21.15.19:0/1380431200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6c0b1f50 0x7f0a6c0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 1 -- 172.21.15.19:0/1380431200 >> 172.21.15.19:0/1380431200 conn(0x7f0a6c06a2d0 msgr2=0x7f0a6c0afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:29.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 1 -- 172.21.15.19:0/1380431200 shutdown_connections 2023-10-06T14:26:29.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 1 -- 172.21.15.19:0/1380431200 wait complete. 2023-10-06T14:26:29.595 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.590+0000 7f0a72651640 1 librados: shutdown 2023-10-06T14:26:29.603 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T14:26:29.603 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T14:26:29.713 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.711+0000 7fc0bfca4640 10 monclient: get_monmap_and_config 2023-10-06T14:26:29.713 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.711+0000 7fc0bfca4640 10 monclient: build_initial_monmap 2023-10-06T14:26:29.713 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.711+0000 7fc0bfca4640 10 monclient: monmap: 2023-10-06T14:26:29.714 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:29.714 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.714 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:29.712337+0000 2023-10-06T14:26:29.714 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:29.712337+0000 2023-10-06T14:26:29.714 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:29.714 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.715 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:29.715 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:29.715 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:29.715 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:29.715 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:29.715 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.718 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.711+0000 7fc0bfca4640 1 Processor -- start 2023-10-06T14:26:29.718 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 1 -- start start 2023-10-06T14:26:29.719 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 10 monclient: init 2023-10-06T14:26:29.719 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:29.719 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2023-10-06T14:26:29.719 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc0b80b3300 addr v1:172.21.15.83:6789/0 2023-10-06T14:26:29.719 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0b8142590 0x7fc0b813bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.719 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 10 monclient(hunting): picked mon.noname-a con 0x7fc0b8142590 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:29.719 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc0b813c040 0x7fc0b8149210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 10 monclient(hunting): picked mon.noname-b con 0x7fc0b813c040 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b80b36d0 con 0x7fc0b80b3300 2023-10-06T14:26:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b80ac180 con 0x7fc0b8142590 2023-10-06T14:26:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80ac360 con 0x7fc0b813c040 2023-10-06T14:26:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.712+0000 7fc0bfca4640 10 monclient(hunting): authenticate will time out at 3317.899086s 2023-10-06T14:26:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.713+0000 7fc0be241640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc0b813c040 0x7fc0b8149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.713+0000 7fc0bd23f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0b8142590 0x7fc0b813bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.713+0000 7fc0bda40640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc0b80b3300 0x7fc0b8141e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:42954/0 (socket says 172.21.15.19:42954) 2023-10-06T14:26:29.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.713+0000 7fc0bda40640 1 -- 172.21.15.19:0/2351252459 learned_addr learned my addr 172.21.15.19:0/2351252459 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T14:26:29.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.713+0000 7fc0be241640 10 monclient(hunting): get_auth_request con 0x7fc0b813c040 auth_method 0 2023-10-06T14:26:29.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.713+0000 7fc0be241640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:29.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.713+0000 7fc0be241640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:29.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.713+0000 7fc0be241640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:29.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.713+0000 7fc0bd23f640 10 monclient(hunting): get_auth_request con 0x7fc0b8142590 auth_method 0 2023-10-06T14:26:29.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.713+0000 7fc0bd23f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:29.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.713+0000 7fc0bd23f640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:29.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.713+0000 7fc0bd23f640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:29.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.714+0000 7fc0be241640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:29.722 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.714+0000 7fc0be241640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:29.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.714+0000 7fc0be241640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:29.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.714+0000 7fc0bd23f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:29.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.714+0000 7fc0bd23f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:29.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.714+0000 7fc0bd23f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:29.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.715+0000 7fc0bca3e640 1 -- 172.21.15.19:0/2351252459 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334838140 0 0) 0x7fc0b80b36d0 con 0x7fc0b80b3300 2023-10-06T14:26:29.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.715+0000 7fc0bca3e640 10 monclient(hunting): _negotiate 2023-10-06T14:26:29.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.715+0000 7fc0bca3e640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:29.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.715+0000 7fc0bca3e640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:29.723 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.715+0000 7fc0bca3e640 10 monclient(hunting): my global_id is 20743 2023-10-06T14:26:29.724 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.715+0000 7fc0bca3e640 1 -- 172.21.15.19:0/2351252459 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a4003520 con 0x7fc0b80b3300 2023-10-06T14:26:29.724 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.715+0000 7fc0be241640 10 monclient(hunting): handle_auth_done global_id 17141 payload 274 2023-10-06T14:26:29.724 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.716+0000 7fc0be241640 1 -- 172.21.15.19:0/2351252459 >> v2:172.21.15.19:3300/0 conn(0x7fc0b8142590 msgr2=0x7fc0b813bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:29.724 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.716+0000 7fc0be241640 1 --2- 172.21.15.19:0/2351252459 >> v2:172.21.15.19:3300/0 conn(0x7fc0b8142590 0x7fc0b813bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.724 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.716+0000 7fc0be241640 1 -- 172.21.15.19:0/2351252459 >> v1:172.21.15.83:6789/0 conn(0x7fc0b80b3300 legacy=0x7fc0b8141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.724 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.716+0000 7fc0be241640 10 monclient: _finish_hunting 0 2023-10-06T14:26:29.724 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.716+0000 7fc0be241640 1 monclient: found mon.noname-b 2023-10-06T14:26:29.724 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.716+0000 7fc0be241640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:29.724 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.716+0000 7fc0be241640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T14:26:29.725 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.716+0000 7fc0be241640 1 -- 172.21.15.19:0/2351252459 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b80ac640 con 0x7fc0b813c040 2023-10-06T14:26:29.729 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.718+0000 7fc0be241640 1 --2- 172.21.15.19:0/2351252459 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc0b813c040 0x7fc0b8149210 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0009d90 tx=0x7fc0b002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d093634bb6840c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:29.729 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bca3e640 1 -- 172.21.15.19:0/2351252459 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b002cc50 con 0x7fc0b813c040 2023-10-06T14:26:29.729 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bca3e640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:29.729 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bca3e640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:29.729 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bca3e640 10 monclient: dump: 2023-10-06T14:26:29.730 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:29.730 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.730 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.730 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.730 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:29.730 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.730 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:29.730 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:29.730 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:29.730 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bca3e640 10 monclient: _finish_auth 0 2023-10-06T14:26:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bca3e640 10 monclient: _check_auth_tickets 2023-10-06T14:26:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bca3e640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bfca4640 5 monclient: authenticate success, global_id 17141 2023-10-06T14:26:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bfca4640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bca3e640 1 -- 172.21.15.19:0/2351252459 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b002cdb0 con 0x7fc0b813c040 2023-10-06T14:26:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bca3e640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bca3e640 1 -- 172.21.15.19:0/2351252459 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0034640 con 0x7fc0b813c040 2023-10-06T14:26:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bca3e640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bca3e640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bca3e640 10 monclient: dump: 2023-10-06T14:26:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:29.733 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bfca4640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:29.734 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.721+0000 7fc0bfca4640 4 set_mon_vals no callback set 2023-10-06T14:26:29.734 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.722+0000 7fc0bfca4640 10 monclient: shutdown 2023-10-06T14:26:29.734 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.722+0000 7fc0bfca4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:29.734 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.722+0000 7fc0bfca4640 1 -- 172.21.15.19:0/2351252459 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc0b813c040 msgr2=0x7fc0b8149210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.734 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.722+0000 7fc0bfca4640 1 --2- 172.21.15.19:0/2351252459 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc0b813c040 0x7fc0b8149210 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0009d90 tx=0x7fc0b002bea0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.734 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.722+0000 7fc0bfca4640 1 -- 172.21.15.19:0/2351252459 shutdown_connections 2023-10-06T14:26:29.734 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.722+0000 7fc0bfca4640 1 --2- 172.21.15.19:0/2351252459 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc0b813c040 0x7fc0b8149210 unknown :-1 s=CLOSED pgs=3122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.735 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.722+0000 7fc0bfca4640 1 --2- 172.21.15.19:0/2351252459 >> v2:172.21.15.19:3300/0 conn(0x7fc0b8142590 0x7fc0b813bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.735 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.722+0000 7fc0bfca4640 1 -- 172.21.15.19:0/2351252459 >> 172.21.15.19:0/2351252459 conn(0x7fc0b8065f50 msgr2=0x7fc0b80b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:29.735 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.722+0000 7fc0bfca4640 1 -- 172.21.15.19:0/2351252459 shutdown_connections 2023-10-06T14:26:29.735 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.722+0000 7fc0bfca4640 1 -- 172.21.15.19:0/2351252459 wait complete. 2023-10-06T14:26:29.735 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.723+0000 7fc0bfca4640 10 monclient: build_initial_monmap 2023-10-06T14:26:29.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.723+0000 7fc0bfca4640 10 monclient: monmap: 2023-10-06T14:26:29.736 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:29.736 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:29.736 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:29.736 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:29.736 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:29.736 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.736 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:29.737 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:29.737 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:29.737 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.723+0000 7fc0bfca4640 1 librados: starting msgr at 2023-10-06T14:26:29.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.723+0000 7fc0bfca4640 1 librados: starting objecter 2023-10-06T14:26:29.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.723+0000 7fc0bfca4640 1 Processor -- start 2023-10-06T14:26:29.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.723+0000 7fc0bfca4640 1 -- start start 2023-10-06T14:26:29.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.723+0000 7fc0bfca4640 1 librados: setting wanted keys 2023-10-06T14:26:29.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.723+0000 7fc0bfca4640 1 librados: calling monclient init 2023-10-06T14:26:29.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.723+0000 7fc0bfca4640 10 monclient: init 2023-10-06T14:26:29.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:29.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T14:26:29.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b80b3300 0x7fc0b81d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 10 monclient(hunting): picked mon.noname-b con 0x7fc0b80b3300 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:29.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0b813c040 0x7fc0b806df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 10 monclient(hunting): picked mon.noname-a con 0x7fc0b813c040 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:29.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc0b8142590 0x7fc0b806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 10 monclient(hunting): picked mon.noname-c con 0x7fc0b8142590 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:29.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:29.738 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b81d3980 con 0x7fc0b813c040 2023-10-06T14:26:29.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:29.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b806e6a0 con 0x7fc0b80b3300 2023-10-06T14:26:29.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:29.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80aa370 con 0x7fc0b8142590 2023-10-06T14:26:29.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:29.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.724+0000 7fc0bfca4640 10 monclient(hunting): authenticate will time out at 3317.910973s 2023-10-06T14:26:29.739 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bda40640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b80b3300 0x7fc0b81d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0be241640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc0b8142590 0x7fc0b806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bd23f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0b813c040 0x7fc0b806df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bda40640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b80b3300 0x7fc0b81d5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:60522/0 (socket says 172.21.15.19:60522) 2023-10-06T14:26:29.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bda40640 1 -- 172.21.15.19:0/3388177708 learned_addr learned my addr 172.21.15.19:0/3388177708 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:29.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bda40640 10 monclient(hunting): get_auth_request con 0x7fc0b80b3300 auth_method 0 2023-10-06T14:26:29.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bda40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:29.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bda40640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:29.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bda40640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:29.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0be241640 10 monclient(hunting): get_auth_request con 0x7fc0b8142590 auth_method 0 2023-10-06T14:26:29.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0be241640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:29.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0be241640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:29.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0be241640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:29.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bd23f640 10 monclient(hunting): get_auth_request con 0x7fc0b813c040 auth_method 0 2023-10-06T14:26:29.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bd23f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:29.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bd23f640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:29.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bd23f640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:29.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bda40640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:29.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bda40640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:29.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.725+0000 7fc0bda40640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:29.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.726+0000 7fc0bd23f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:29.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.726+0000 7fc0bd23f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:29.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.726+0000 7fc0bd23f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:29.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.726+0000 7fc0be241640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:29.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.726+0000 7fc0be241640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:29.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.726+0000 7fc0be241640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:29.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.728+0000 7fc0bda40640 10 monclient(hunting): handle_auth_done global_id 20746 payload 824 2023-10-06T14:26:29.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.728+0000 7fc0bda40640 1 -- 172.21.15.19:0/3388177708 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc0b8142590 msgr2=0x7fc0b806eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.728+0000 7fc0bda40640 1 --2- 172.21.15.19:0/3388177708 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc0b8142590 0x7fc0b806eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.728+0000 7fc0bda40640 1 -- 172.21.15.19:0/3388177708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0b813c040 msgr2=0x7fc0b806df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.728+0000 7fc0bda40640 1 --2- 172.21.15.19:0/3388177708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0b813c040 0x7fc0b806df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.728+0000 7fc0bda40640 10 monclient: _finish_hunting 0 2023-10-06T14:26:29.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.728+0000 7fc0bda40640 1 monclient: found mon.noname-b 2023-10-06T14:26:29.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.728+0000 7fc0bda40640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:29.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.728+0000 7fc0bda40640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T14:26:29.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.728+0000 7fc0bda40640 1 -- 172.21.15.19:0/3388177708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b0009a70 con 0x7fc0b80b3300 2023-10-06T14:26:29.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.730+0000 7fc0bda40640 1 --2- 172.21.15.19:0/3388177708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b80b3300 0x7fc0b81d5f30 secure :-1 s=READY pgs=5109 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400c500 tx=0x7fc0b400c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:29.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.730+0000 7fc0ae7fc640 10 client.?.objecter ms_handle_connect 0x7fc0b80b3300 2023-10-06T14:26:29.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.730+0000 7fc0ae7fc640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:29.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0ae7fc640 1 -- 172.21.15.19:0/3388177708 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4005ce0 con 0x7fc0b80b3300 2023-10-06T14:26:29.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:29.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0ae7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:29.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0ae7fc640 10 monclient: dump: 2023-10-06T14:26:29.748 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:29.748 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.749 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.749 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.749 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:29.749 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.749 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:29.749 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:29.749 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:29.749 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0ae7fc640 10 monclient: _finish_auth 0 2023-10-06T14:26:29.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0ae7fc640 10 monclient: _check_auth_tickets 2023-10-06T14:26:29.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:29.750 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0ae7fc640 1 -- 172.21.15.19:0/3388177708 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b4005e40 con 0x7fc0b80b3300 2023-10-06T14:26:29.750 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0bfca4640 5 monclient: authenticate success, global_id 20746 2023-10-06T14:26:29.750 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0bfca4640 10 monclient: _renew_subs 2023-10-06T14:26:29.750 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0bfca4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:29.750 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0bfca4640 1 -- 172.21.15.19:0/3388177708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b81d6950 con 0x7fc0b80b3300 2023-10-06T14:26:29.750 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0ae7fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:29.750 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0ae7fc640 1 -- 172.21.15.19:0/3388177708 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4007330 con 0x7fc0b80b3300 2023-10-06T14:26:29.750 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:29.750 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0ae7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:29.750 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0ae7fc640 10 monclient: dump: 2023-10-06T14:26:29.750 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:29.750 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.751 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.751 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.751 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:29.751 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.751 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:29.751 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:29.751 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:29.751 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.751 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0bfca4640 10 client.20746.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:29.751 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0affff640 4 set_mon_vals no callback set 2023-10-06T14:26:29.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0bfca4640 10 monclient: _renew_subs 2023-10-06T14:26:29.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0bfca4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:29.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0bfca4640 1 -- 172.21.15.19:0/3388177708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b80a8630 con 0x7fc0b80b3300 2023-10-06T14:26:29.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.734+0000 7fc0bfca4640 1 librados: init done 2023-10-06T14:26:29.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.735+0000 7fc0bfca4640 10 client.20746.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:29.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.735+0000 7fc0bfca4640 15 client.20746.objecter _session_command_op_assign -1 1 2023-10-06T14:26:29.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.735+0000 7fc0bfca4640 20 client.20746.objecter _get_session osd=-1 returning homeless 2023-10-06T14:26:29.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.735+0000 7fc0bfca4640 10 client.20746.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:29.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.735+0000 7fc0bfca4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:29.752 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.735+0000 7fc0bfca4640 1 -- 172.21.15.19:0/3388177708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc0b80b2710 con 0x7fc0b80b3300 2023-10-06T14:26:29.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.754+0000 7fc0ae7fc640 1 -- 172.21.15.19:0/3388177708 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc0b4007490 con 0x7fc0b80b3300 2023-10-06T14:26:29.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.755+0000 7fc0ae7fc640 1 --2- 172.21.15.19:0/3388177708 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc09c06c1f0 0x7fc09c06e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.755+0000 7fc0bd23f640 1 --2- 172.21.15.19:0/3388177708 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc09c06c1f0 0x7fc09c06e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.755+0000 7fc0bd23f640 10 monclient: get_auth_request con 0x7fc09c06c1f0 auth_method 0 2023-10-06T14:26:29.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.756+0000 7fc0bd23f640 1 --2- 172.21.15.19:0/3388177708 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc09c06c1f0 0x7fc09c06e6a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc0b806d900 tx=0x7fc0a8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:29.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.756+0000 7fc0ae7fc640 10 client.20746.objecter ms_handle_connect 0x7fc09c06c1f0 2023-10-06T14:26:29.760 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 1 -- 172.21.15.19:0/3388177708 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(970..970 src has 595..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fc0b4007740 con 0x7fc0b80b3300 2023-10-06T14:26:29.760 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 10 client.20746.objecter ms_dispatch 0x7fc0b8000b90 osd_map(970..970 src has 595..970) v4 2023-10-06T14:26:29.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 3 client.20746.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:29.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 3 client.20746.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:29.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 10 client.20746.objecter checking command 1 2023-10-06T14:26:29.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 1 --2- 172.21.15.19:0/3388177708 >> [v2:172.21.15.19:6817/86193,v1:172.21.15.19:6819/86193] conn(0x7fc09c073a10 0x7fc09c075ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 20 client.20746.objecter _get_session s=0x7fc09c073340 osd=1 3 2023-10-06T14:26:29.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 20 client.20746.objecter put_session s=0x7fc09c073340 osd=1 3 2023-10-06T14:26:29.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 15 client.20746.objecter _session_command_op_remove -1 1 2023-10-06T14:26:29.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 20 client.20746.objecter _get_session s=0x7fc09c073340 osd=1 3 2023-10-06T14:26:29.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 20 client.20746.objecter get_session s=0x7fc09c073340 osd=1 3 2023-10-06T14:26:29.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 15 client.20746.objecter _session_command_op_assign 1 1 2023-10-06T14:26:29.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 20 client.20746.objecter put_session s=0x7fc09c073340 osd=1 4 2023-10-06T14:26:29.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 10 client.20746.objecter _send_command 1 2023-10-06T14:26:29.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 1 -- 172.21.15.19:0/3388177708 --> [v2:172.21.15.19:6817/86193,v1:172.21.15.19:6819/86193] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc09c076570 con 0x7fc09c073a10 2023-10-06T14:26:29.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 20 client.20746.objecter dump_active .. 0 homeless 2023-10-06T14:26:29.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 1 -- 172.21.15.19:0/3388177708 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=970) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0b4057b30 con 0x7fc0b80b3300 2023-10-06T14:26:29.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.757+0000 7fc0ae7fc640 10 monclient: handle_get_version_reply finishing 1 version 970 2023-10-06T14:26:29.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.758+0000 7fc0be241640 1 --2- 172.21.15.19:0/3388177708 >> [v2:172.21.15.19:6817/86193,v1:172.21.15.19:6819/86193] conn(0x7fc09c073a10 0x7fc09c075ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.758+0000 7fc0be241640 10 monclient: get_auth_request con 0x7fc09c073a10 auth_method 0 2023-10-06T14:26:29.764 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.762+0000 7fc0ae7fc640 10 client.20746.objecter ms_handle_connect 0x7fc09c073a10 2023-10-06T14:26:29.764 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.762+0000 7fc0be241640 1 --2- 172.21.15.19:0/3388177708 >> [v2:172.21.15.19:6817/86193,v1:172.21.15.19:6819/86193] conn(0x7fc09c073a10 0x7fc09c075ea0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:29.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.786+0000 7fc0ae7fc640 1 -- 172.21.15.19:0/3388177708 <== osd.1 v2:172.21.15.19:6817/86193 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc09c076570 con 0x7fc09c073a10 2023-10-06T14:26:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.786+0000 7fc0ae7fc640 10 client.20746.objecter ms_dispatch 0x7fc0b8000b90 command_reply(tid 1: 0 ) v1 2023-10-06T14:26:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.786+0000 7fc0ae7fc640 10 client.20746.objecter _finish_command 1 = system:0 2023-10-06T14:26:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.786+0000 7fc0ae7fc640 20 client.20746.objecter put_session s=0x7fc09c073340 osd=1 4 2023-10-06T14:26:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.786+0000 7fc0ae7fc640 15 client.20746.objecter _session_command_op_remove 1 1 2023-10-06T14:26:29.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.799+0000 7fc0bfca4640 10 client.20746.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-10-06T14:26:29.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.799+0000 7fc0bfca4640 15 client.20746.objecter _session_command_op_assign -1 2 2023-10-06T14:26:29.801 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.799+0000 7fc0bfca4640 20 client.20746.objecter _get_session s=0x7fc09c073340 osd=1 3 2023-10-06T14:26:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.799+0000 7fc0bfca4640 20 client.20746.objecter put_session s=0x7fc09c073340 osd=1 3 2023-10-06T14:26:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.799+0000 7fc0bfca4640 20 client.20746.objecter _get_session s=0x7fc09c073340 osd=1 3 2023-10-06T14:26:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.799+0000 7fc0bfca4640 15 client.20746.objecter _session_command_op_remove -1 2 2023-10-06T14:26:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.799+0000 7fc0bfca4640 20 client.20746.objecter get_session s=0x7fc09c073340 osd=1 3 2023-10-06T14:26:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.799+0000 7fc0bfca4640 15 client.20746.objecter _session_command_op_assign 1 2 2023-10-06T14:26:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.799+0000 7fc0bfca4640 20 client.20746.objecter put_session s=0x7fc09c073340 osd=1 4 2023-10-06T14:26:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.799+0000 7fc0bfca4640 10 client.20746.objecter _send_command 2 2023-10-06T14:26:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.799+0000 7fc0bfca4640 1 -- 172.21.15.19:0/3388177708 --> [v2:172.21.15.19:6817/86193,v1:172.21.15.19:6819/86193] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc0b80aaa20 con 0x7fc09c073a10 2023-10-06T14:26:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0ae7fc640 1 -- 172.21.15.19:0/3388177708 <== osd.1 v2:172.21.15.19:6817/86193 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc0b0009450 con 0x7fc09c073a10 2023-10-06T14:26:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0ae7fc640 10 client.20746.objecter ms_dispatch 0x7fc0b8000b90 command_reply(tid 2: 0 ) v1 2023-10-06T14:26:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0ae7fc640 10 client.20746.objecter _finish_command 2 = system:0 2023-10-06T14:26:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0ae7fc640 20 client.20746.objecter put_session s=0x7fc09c073340 osd=1 4 2023-10-06T14:26:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0ae7fc640 15 client.20746.objecter _session_command_op_remove 1 2 2023-10-06T14:26:29.817 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2023-10-06T14:26:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T14:26:29.817 INFO:teuthology.orchestra.run.smithi019.stdout:} 2023-10-06T14:26:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 10 librados: watch_flush enter 2023-10-06T14:26:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 10 librados: watch_flush exit 2023-10-06T14:26:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 10 client.20746.objecter close_session for osd.1 2023-10-06T14:26:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 1 -- 172.21.15.19:0/3388177708 >> [v2:172.21.15.19:6817/86193,v1:172.21.15.19:6819/86193] conn(0x7fc09c073a10 msgr2=0x7fc09c075ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 1 --2- 172.21.15.19:0/3388177708 >> [v2:172.21.15.19:6817/86193,v1:172.21.15.19:6819/86193] conn(0x7fc09c073a10 0x7fc09c075ea0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 20 client.20746.objecter put_session s=0x7fc09c073340 osd=1 1 2023-10-06T14:26:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 20 client.20746.objecter shutdown clearing up homeless session... 2023-10-06T14:26:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 10 client.20746.objecter successfully canceled tick 2023-10-06T14:26:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 1 -- 172.21.15.19:0/3388177708 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc09c06c1f0 msgr2=0x7fc09c06e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 1 --2- 172.21.15.19:0/3388177708 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fc09c06c1f0 0x7fc09c06e6a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc0b806d900 tx=0x7fc0a8009190 comp rx=0 tx=0).stop 2023-10-06T14:26:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 10 monclient: shutdown 2023-10-06T14:26:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 1 -- 172.21.15.19:0/3388177708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b80b3300 msgr2=0x7fc0b81d5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 1 --2- 172.21.15.19:0/3388177708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b80b3300 0x7fc0b81d5f30 secure :-1 s=READY pgs=5109 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400c500 tx=0x7fc0b400c9c0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0be241640 1 -- 172.21.15.19:0/3388177708 reap_dead start 2023-10-06T14:26:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 1 -- 172.21.15.19:0/3388177708 shutdown_connections 2023-10-06T14:26:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 1 -- 172.21.15.19:0/3388177708 >> 172.21.15.19:0/3388177708 conn(0x7fc0b8065f50 msgr2=0x7fc0b8068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 1 -- 172.21.15.19:0/3388177708 shutdown_connections 2023-10-06T14:26:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 1 -- 172.21.15.19:0/3388177708 wait complete. 2023-10-06T14:26:29.821 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.814+0000 7fc0bfca4640 1 librados: shutdown 2023-10-06T14:26:29.827 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T14:26:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.932+0000 7fda03fc5640 10 monclient: get_monmap_and_config 2023-10-06T14:26:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.932+0000 7fda03fc5640 10 monclient: build_initial_monmap 2023-10-06T14:26:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.932+0000 7fda03fc5640 10 monclient: monmap: 2023-10-06T14:26:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:29.933403+0000 2023-10-06T14:26:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:29.933403+0000 2023-10-06T14:26:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:29.935 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.932+0000 7fda03fc5640 1 Processor -- start 2023-10-06T14:26:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.933+0000 7fda03fc5640 1 -- start start 2023-10-06T14:26:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.933+0000 7fda03fc5640 10 monclient: init 2023-10-06T14:26:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.933+0000 7fda03fc5640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.933+0000 7fda03fc5640 10 monclient: _add_conns ranks=[0,2,1,3,4] 2023-10-06T14:26:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.933+0000 7fda03fc5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd9fc0b39e0 addr v1:172.21.15.19:6789/0 2023-10-06T14:26:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.933+0000 7fda03fc5640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd9fc13e6f0 0x7fd9fc0b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.933+0000 7fda03fc5640 10 monclient(hunting): picked mon.noname-b con 0x7fd9fc13e6f0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.933+0000 7fda03fc5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd9fc0b3db0 addr v1:172.21.15.83:6789/0 2023-10-06T14:26:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.933+0000 7fda03fc5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9fc0b16e0 con 0x7fd9fc0b39e0 2023-10-06T14:26:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.933+0000 7fda03fc5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9fc0af020 con 0x7fd9fc0b3db0 2023-10-06T14:26:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.933+0000 7fda03fc5640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.933+0000 7fda03fc5640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc0b31f0 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.933+0000 7fda03fc5640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.933+0000 7fda03fc5640 10 monclient(hunting): authenticate will time out at 3318.120145s 2023-10-06T14:26:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.934+0000 7fda01560640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd9fc13e6f0 0x7fd9fc0b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.934+0000 7fda01d61640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd9fc0b39e0 0x7fd9fc13dfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48034/0 (socket says 172.21.15.19:48034) 2023-10-06T14:26:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.934+0000 7fda01d61640 1 -- 172.21.15.19:0/3895533638 learned_addr learned my addr 172.21.15.19:0/3895533638 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T14:26:29.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.934+0000 7fda01560640 10 monclient(hunting): get_auth_request con 0x7fd9fc13e6f0 auth_method 0 2023-10-06T14:26:29.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.934+0000 7fda01560640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:29.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.934+0000 7fda01560640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:29.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.934+0000 7fda01560640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:29.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.935+0000 7fda01560640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:29.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.935+0000 7fda01560640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:29.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.935+0000 7fda01560640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:29.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.937+0000 7fda01560640 10 monclient(hunting): handle_auth_done global_id 17147 payload 274 2023-10-06T14:26:29.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.937+0000 7fda01560640 1 -- 172.21.15.19:0/3895533638 >> v1:172.21.15.83:6789/0 conn(0x7fd9fc0b3db0 legacy=0x7fd9fc1446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.937+0000 7fda01560640 1 -- 172.21.15.19:0/3895533638 >> v1:172.21.15.19:6789/0 conn(0x7fd9fc0b39e0 legacy=0x7fd9fc13dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.937+0000 7fda01560640 10 monclient: _finish_hunting 0 2023-10-06T14:26:29.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.937+0000 7fda01560640 1 monclient: found mon.noname-b 2023-10-06T14:26:29.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.937+0000 7fda00d5f640 1 -- 172.21.15.19:0/3895533638 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896336641 0 0) 0x7fd9fc0af020 con 0x7fd9fc0b3db0 2023-10-06T14:26:29.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.937+0000 7fda01560640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:29.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.937+0000 7fda01560640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T14:26:29.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.937+0000 7fda01560640 1 -- 172.21.15.19:0/3895533638 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc144db0 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.937+0000 7fda00d5f640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2023-10-06T14:26:29.942 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.940+0000 7fda01560640 1 --2- 172.21.15.19:0/3895533638 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd9fc13e6f0 0x7fd9fc0b2bd0 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0097d0 tx=0x7fd9ec02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c73272916538ef60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:29.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda00d5f640 1 -- 172.21.15.19:0/3895533638 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec031890 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda00d5f640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:29.947 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda00d5f640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:29.947 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda00d5f640 10 monclient: dump: 2023-10-06T14:26:29.947 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:29.947 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.947 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.947 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.947 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:29.947 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.947 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:29.947 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:29.947 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:29.948 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda00d5f640 10 monclient: _finish_auth 0 2023-10-06T14:26:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda00d5f640 10 monclient: _check_auth_tickets 2023-10-06T14:26:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda00d5f640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda00d5f640 1 -- 172.21.15.19:0/3895533638 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ec0319f0 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda00d5f640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda00d5f640 1 -- 172.21.15.19:0/3895533638 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec031d80 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda03fc5640 5 monclient: authenticate success, global_id 17147 2023-10-06T14:26:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda03fc5640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda03fc5640 4 set_mon_vals no callback set 2023-10-06T14:26:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda00d5f640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda00d5f640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.944+0000 7fda00d5f640 10 monclient: dump: 2023-10-06T14:26:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:29.950 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:29.950 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:29.950 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.945+0000 7fda03fc5640 10 monclient: shutdown 2023-10-06T14:26:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.945+0000 7fda03fc5640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.945+0000 7fda03fc5640 1 -- 172.21.15.19:0/3895533638 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd9fc13e6f0 msgr2=0x7fd9fc0b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.945+0000 7fda03fc5640 1 --2- 172.21.15.19:0/3895533638 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd9fc13e6f0 0x7fd9fc0b2bd0 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0097d0 tx=0x7fd9ec02b930 comp rx=0 tx=0).stop 2023-10-06T14:26:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.945+0000 7fda03fc5640 1 -- 172.21.15.19:0/3895533638 shutdown_connections 2023-10-06T14:26:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.945+0000 7fda03fc5640 1 --2- 172.21.15.19:0/3895533638 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd9fc13e6f0 0x7fd9fc0b2bd0 unknown :-1 s=CLOSED pgs=3123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.945+0000 7fda03fc5640 1 -- 172.21.15.19:0/3895533638 >> 172.21.15.19:0/3895533638 conn(0x7fd9fc065920 msgr2=0x7fd9fc065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.945+0000 7fda03fc5640 1 -- 172.21.15.19:0/3895533638 shutdown_connections 2023-10-06T14:26:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.945+0000 7fda03fc5640 1 -- 172.21.15.19:0/3895533638 wait complete. 2023-10-06T14:26:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.946+0000 7fda03fc5640 10 monclient: build_initial_monmap 2023-10-06T14:26:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.946+0000 7fda03fc5640 10 monclient: monmap: 2023-10-06T14:26:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:29.956 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.946+0000 7fda03fc5640 1 librados: starting msgr at 2023-10-06T14:26:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.946+0000 7fda03fc5640 1 librados: starting objecter 2023-10-06T14:26:29.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.946+0000 7fda03fc5640 1 Processor -- start 2023-10-06T14:26:29.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.946+0000 7fda03fc5640 1 -- start start 2023-10-06T14:26:29.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.946+0000 7fda03fc5640 1 librados: setting wanted keys 2023-10-06T14:26:29.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.946+0000 7fda03fc5640 1 librados: calling monclient init 2023-10-06T14:26:29.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.946+0000 7fda03fc5640 10 monclient: init 2023-10-06T14:26:29.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:29.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T14:26:29.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9fc0b39e0 0x7fd9fc139de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 10 monclient(hunting): picked mon.noname-b con 0x7fd9fc0b39e0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:29.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9fc0b3db0 0x7fd9fc13fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 10 monclient(hunting): picked mon.noname-a con 0x7fd9fc0b3db0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:29.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd9fc13e6f0 0x7fd9fc1d96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 10 monclient(hunting): picked mon.noname-c con 0x7fd9fc13e6f0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:29.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:29.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc13d130 con 0x7fd9fc0b3db0 2023-10-06T14:26:29.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:29.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc13b680 con 0x7fd9fc0b39e0 2023-10-06T14:26:29.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:29.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc13b960 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:29.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda03fc5640 10 monclient(hunting): authenticate will time out at 3318.133500s 2023-10-06T14:26:29.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda02562640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd9fc13e6f0 0x7fd9fc1d96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda01560640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9fc0b3db0 0x7fd9fc13fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda02562640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd9fc13e6f0 0x7fd9fc1d96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40310/0 (socket says 172.21.15.19:40310) 2023-10-06T14:26:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda02562640 1 -- 172.21.15.19:0/801747297 learned_addr learned my addr 172.21.15.19:0/801747297 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda02562640 10 monclient(hunting): get_auth_request con 0x7fd9fc13e6f0 auth_method 0 2023-10-06T14:26:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda02562640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda02562640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda02562640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.947+0000 7fda01d61640 1 --2- 172.21.15.19:0/801747297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9fc0b39e0 0x7fd9fc139de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda01560640 10 monclient(hunting): get_auth_request con 0x7fd9fc0b3db0 auth_method 0 2023-10-06T14:26:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda01560640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda01560640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda01560640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda01d61640 10 monclient(hunting): get_auth_request con 0x7fd9fc0b39e0 auth_method 0 2023-10-06T14:26:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda01d61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda01d61640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda01d61640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda02562640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda02562640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda02562640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda01560640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda01560640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda01560640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda01d61640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda01d61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.948+0000 7fda01d61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.951+0000 7fda02562640 10 monclient(hunting): handle_auth_done global_id 17150 payload 824 2023-10-06T14:26:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.951+0000 7fda02562640 1 -- 172.21.15.19:0/801747297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9fc0b39e0 msgr2=0x7fd9fc139de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.951+0000 7fda02562640 1 --2- 172.21.15.19:0/801747297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9fc0b39e0 0x7fd9fc139de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.951+0000 7fda02562640 1 -- 172.21.15.19:0/801747297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9fc0b3db0 msgr2=0x7fd9fc13fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.951+0000 7fda02562640 1 --2- 172.21.15.19:0/801747297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9fc0b3db0 0x7fd9fc13fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.951+0000 7fda02562640 10 monclient: _finish_hunting 0 2023-10-06T14:26:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.951+0000 7fda02562640 1 monclient: found mon.noname-c 2023-10-06T14:26:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.951+0000 7fda02562640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.951+0000 7fda02562640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T14:26:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.951+0000 7fda02562640 1 -- 172.21.15.19:0/801747297 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec009400 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.953+0000 7fda02562640 1 --2- 172.21.15.19:0/801747297 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd9fc13e6f0 0x7fd9fc1d96f0 secure :-1 s=READY pgs=3124 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c430 tx=0x7fd9f400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.953+0000 7fd9f27fc640 10 client.?.objecter ms_handle_connect 0x7fd9fc13e6f0 2023-10-06T14:26:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.953+0000 7fd9f27fc640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fd9f27fc640 1 -- 172.21.15.19:0/801747297 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40035f0 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fd9f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fd9f27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T14:26:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fd9f27fc640 10 monclient: dump: 2023-10-06T14:26:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:29.965 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fd9f27fc640 10 monclient: _finish_auth 0 2023-10-06T14:26:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fd9f27fc640 10 monclient: _check_auth_tickets 2023-10-06T14:26:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fd9f27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fd9f27fc640 1 -- 172.21.15.19:0/801747297 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f4003770 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fda03fc5640 5 monclient: authenticate success, global_id 17150 2023-10-06T14:26:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fda03fc5640 10 monclient: _renew_subs 2023-10-06T14:26:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fda03fc5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fda03fc5640 1 -- 172.21.15.19:0/801747297 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9fc141640 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fd9f27fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fd9f27fc640 1 -- 172.21.15.19:0/801747297 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4003b20 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fd9f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fd9f27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fd9f27fc640 10 monclient: dump: 2023-10-06T14:26:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:29.967 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fd9f3fff640 4 set_mon_vals no callback set 2023-10-06T14:26:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fda03fc5640 10 client.17150.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fda03fc5640 10 monclient: _renew_subs 2023-10-06T14:26:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fda03fc5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fda03fc5640 1 -- 172.21.15.19:0/801747297 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9fc140230 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.957+0000 7fda03fc5640 1 librados: init done 2023-10-06T14:26:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.958+0000 7fda03fc5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.958+0000 7fda03fc5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.958+0000 7fda03fc5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.958+0000 7fda03fc5640 1 -- 172.21.15.19:0/801747297 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9fc13a320 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.978+0000 7fd9f27fc640 1 -- 172.21.15.19:0/801747297 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd9f4003c80 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.979+0000 7fd9f27fc640 1 --2- 172.21.15.19:0/801747297 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd9e006c1f0 0x7fd9e006e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.979+0000 7fda01d61640 1 --2- 172.21.15.19:0/801747297 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd9e006c1f0 0x7fd9e006e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.979+0000 7fda01d61640 10 monclient: get_auth_request con 0x7fd9e006c1f0 auth_method 0 2023-10-06T14:26:29.982 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.979+0000 7fda01d61640 1 --2- 172.21.15.19:0/801747297 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd9e006c1f0 0x7fd9e006e6a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80012e0 tx=0x7fd9f8001a30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:29.982 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.979+0000 7fd9f27fc640 10 client.17150.objecter ms_handle_connect 0x7fd9e006c1f0 2023-10-06T14:26:29.984 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.981+0000 7fd9f27fc640 1 -- 172.21.15.19:0/801747297 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(970..970 src has 595..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fd9f40579b0 con 0x7fd9fc13e6f0 2023-10-06T14:26:29.984 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.981+0000 7fd9f27fc640 10 client.17150.objecter ms_dispatch 0x7fd9fc000b90 osd_map(970..970 src has 595..970) v4 2023-10-06T14:26:29.984 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.981+0000 7fd9f27fc640 3 client.17150.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:29.984 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.981+0000 7fd9f27fc640 3 client.17150.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:29.984 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:29.982+0000 7fd9f27fc640 20 client.17150.objecter dump_active .. 0 homeless 2023-10-06T14:26:30.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.232+0000 7fd9f27fc640 1 -- 172.21.15.19:0/801747297 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7fd9f4053cf0 con 0x7fd9fc13e6f0 2023-10-06T14:26:30.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.232+0000 7fd9f27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:30.234 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.232+0000 7fd9f27fc640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:26:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.415+0000 7fda03fc5640 1 -- 172.21.15.19:0/801747297 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9fc145130 con 0x7fd9e006c1f0 2023-10-06T14:26:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.416+0000 7fd9f27fc640 1 -- 172.21.15.19:0/801747297 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd9f8001f60 con 0x7fd9e006c1f0 2023-10-06T14:26:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T14:26:30.420 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.417+0000 7fda03fc5640 10 librados: watch_flush enter 2023-10-06T14:26:30.420 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.417+0000 7fda03fc5640 10 librados: watch_flush exit 2023-10-06T14:26:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.417+0000 7fda03fc5640 20 client.17150.objecter shutdown clearing up homeless session... 2023-10-06T14:26:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.417+0000 7fda03fc5640 10 client.17150.objecter successfully canceled tick 2023-10-06T14:26:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 1 -- 172.21.15.19:0/801747297 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd9e006c1f0 msgr2=0x7fd9e006e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 1 --2- 172.21.15.19:0/801747297 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd9e006c1f0 0x7fd9e006e6a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80012e0 tx=0x7fd9f8001a30 comp rx=0 tx=0).stop 2023-10-06T14:26:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 10 monclient: shutdown 2023-10-06T14:26:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 1 -- 172.21.15.19:0/801747297 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd9fc13e6f0 msgr2=0x7fd9fc1d96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 1 --2- 172.21.15.19:0/801747297 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd9fc13e6f0 0x7fd9fc1d96f0 secure :-1 s=READY pgs=3124 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c430 tx=0x7fd9f400c8f0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 1 -- 172.21.15.19:0/801747297 shutdown_connections 2023-10-06T14:26:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 1 --2- 172.21.15.19:0/801747297 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fd9e006c1f0 0x7fd9e006e6a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 1 --2- 172.21.15.19:0/801747297 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fd9fc13e6f0 0x7fd9fc1d96f0 unknown :-1 s=CLOSED pgs=3124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 1 --2- 172.21.15.19:0/801747297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9fc0b3db0 0x7fd9fc13fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 1 --2- 172.21.15.19:0/801747297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9fc0b39e0 0x7fd9fc139de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 1 -- 172.21.15.19:0/801747297 >> 172.21.15.19:0/801747297 conn(0x7fd9fc065920 msgr2=0x7fd9fc0b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 1 -- 172.21.15.19:0/801747297 shutdown_connections 2023-10-06T14:26:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 1 -- 172.21.15.19:0/801747297 wait complete. 2023-10-06T14:26:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.418+0000 7fda03fc5640 1 librados: shutdown 2023-10-06T14:26:30.432 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T14:26:30.433 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T14:26:30.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.540+0000 7f7bdd232640 10 monclient: get_monmap_and_config 2023-10-06T14:26:30.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.540+0000 7f7bdd232640 10 monclient: build_initial_monmap 2023-10-06T14:26:30.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.540+0000 7f7bdd232640 10 monclient: monmap: 2023-10-06T14:26:30.542 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:30.542 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:30.542 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:30.541051+0000 2023-10-06T14:26:30.542 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:30.541051+0000 2023-10-06T14:26:30.542 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:30.543 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:30.543 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:30.543 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:30.543 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:30.543 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:30.543 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:30.543 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:30.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.540+0000 7f7bdd232640 1 Processor -- start 2023-10-06T14:26:30.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.540+0000 7f7bdd232640 1 -- start start 2023-10-06T14:26:30.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.540+0000 7f7bdd232640 10 monclient: init 2023-10-06T14:26:30.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bdd232640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:30.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bdd232640 10 monclient: _add_conns ranks=[1,2,0,3,4] 2023-10-06T14:26:30.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bdd232640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7bd80b3990 addr v1:172.21.15.83:6789/0 2023-10-06T14:26:30.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bdd232640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7bd80b3d60 0x7f7bd80af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bdd232640 10 monclient(hunting): picked mon.noname-b con 0x7f7bd80b3d60 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:30.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bdd232640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7bd80b00c0 addr v1:172.21.15.19:6789/0 2023-10-06T14:26:30.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bdd232640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd80aebe0 con 0x7f7bd80b00c0 2023-10-06T14:26:30.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bdd232640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd8141070 con 0x7f7bd80b3990 2023-10-06T14:26:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bdd232640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bdd232640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd8140dd0 con 0x7f7bd80b3d60 2023-10-06T14:26:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bdd232640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bdd232640 10 monclient(hunting): authenticate will time out at 3318.727626s 2023-10-06T14:26:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bd659c640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7bd80b3d60 0x7f7bd80af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bd759e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7bd80b00c0 0x7f7bd814b600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48048/0 (socket says 172.21.15.19:48048) 2023-10-06T14:26:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.541+0000 7f7bd759e640 1 -- 172.21.15.19:0/2284203736 learned_addr learned my addr 172.21.15.19:0/2284203736 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T14:26:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.542+0000 7f7bd659c640 10 monclient(hunting): get_auth_request con 0x7f7bd80b3d60 auth_method 0 2023-10-06T14:26:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.542+0000 7f7bd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.542+0000 7f7bd659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.542+0000 7f7bd659c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.542+0000 7f7bd659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.542+0000 7f7bd659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.542+0000 7f7bd659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.544+0000 7f7bd659c640 10 monclient(hunting): handle_auth_done global_id 17153 payload 274 2023-10-06T14:26:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.544+0000 7f7bd659c640 1 -- 172.21.15.19:0/2284203736 >> v1:172.21.15.83:6789/0 conn(0x7f7bd80b3990 legacy=0x7f7bd8141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.544+0000 7f7bd659c640 1 -- 172.21.15.19:0/2284203736 >> v1:172.21.15.19:6789/0 conn(0x7f7bd80b00c0 legacy=0x7f7bd814b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.544+0000 7f7bd659c640 10 monclient: _finish_hunting 0 2023-10-06T14:26:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.544+0000 7f7bd659c640 1 monclient: found mon.noname-b 2023-10-06T14:26:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.544+0000 7f7bd659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.544+0000 7f7bd659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T14:26:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.544+0000 7f7bd659c640 1 -- 172.21.15.19:0/2284203736 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd8141c10 con 0x7f7bd80b3d60 2023-10-06T14:26:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.546+0000 7f7bd659c640 1 --2- 172.21.15.19:0/2284203736 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7bd80b3d60 0x7f7bd80af810 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00097d0 tx=0x7f7bc002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42dbb1323f2f19a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:30.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bd5d9b640 1 -- 172.21.15.19:0/2284203736 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0031800 con 0x7f7bd80b3d60 2023-10-06T14:26:30.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bd5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:30.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bd5d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:30.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bd5d9b640 10 monclient: dump: 2023-10-06T14:26:30.552 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:30.552 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:30.552 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.552 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.553 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:30.553 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:30.553 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:30.553 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:30.553 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:30.553 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:30.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bd5d9b640 10 monclient: _finish_auth 0 2023-10-06T14:26:30.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bd5d9b640 10 monclient: _check_auth_tickets 2023-10-06T14:26:30.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bd5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:30.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bdd232640 5 monclient: authenticate success, global_id 17153 2023-10-06T14:26:30.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bdd232640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:30.553 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bd5d9b640 1 -- 172.21.15.19:0/2284203736 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc0031960 con 0x7f7bd80b3d60 2023-10-06T14:26:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bd5d9b640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bdd232640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bdd232640 4 set_mon_vals no callback set 2023-10-06T14:26:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bd5d9b640 1 -- 172.21.15.19:0/2284203736 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0031cf0 con 0x7f7bd80b3d60 2023-10-06T14:26:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bd5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bd5d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bd5d9b640 10 monclient: dump: 2023-10-06T14:26:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:30.555 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:30.555 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:30.555 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:30.555 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:30.557 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bdd232640 10 monclient: shutdown 2023-10-06T14:26:30.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bdd232640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:30.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bdd232640 1 -- 172.21.15.19:0/2284203736 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7bd80b3d60 msgr2=0x7f7bd80af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bdd232640 1 --2- 172.21.15.19:0/2284203736 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7bd80b3d60 0x7f7bd80af810 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00097d0 tx=0x7f7bc002b8a0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bdd232640 1 -- 172.21.15.19:0/2284203736 shutdown_connections 2023-10-06T14:26:30.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bdd232640 1 --2- 172.21.15.19:0/2284203736 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7bd80b3d60 0x7f7bd80af810 unknown :-1 s=CLOSED pgs=3125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bdd232640 1 -- 172.21.15.19:0/2284203736 >> 172.21.15.19:0/2284203736 conn(0x7f7bd80b6e00 msgr2=0x7f7bd80b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:30.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.550+0000 7f7bdd232640 1 -- 172.21.15.19:0/2284203736 shutdown_connections 2023-10-06T14:26:30.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.551+0000 7f7bdd232640 1 -- 172.21.15.19:0/2284203736 wait complete. 2023-10-06T14:26:30.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.551+0000 7f7bdd232640 10 monclient: build_initial_monmap 2023-10-06T14:26:30.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.551+0000 7f7bdd232640 10 monclient: monmap: 2023-10-06T14:26:30.558 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:30.559 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:30.559 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:30.559 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:30.559 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:30.559 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:30.559 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:30.559 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:30.559 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:30.559 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:30.559 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.552+0000 7f7bdd232640 1 librados: starting msgr at 2023-10-06T14:26:30.559 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.552+0000 7f7bdd232640 1 librados: starting objecter 2023-10-06T14:26:30.559 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.552+0000 7f7bdd232640 1 Processor -- start 2023-10-06T14:26:30.559 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.552+0000 7f7bdd232640 1 -- start start 2023-10-06T14:26:30.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.552+0000 7f7bdd232640 1 librados: setting wanted keys 2023-10-06T14:26:30.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.552+0000 7f7bdd232640 1 librados: calling monclient init 2023-10-06T14:26:30.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.552+0000 7f7bdd232640 10 monclient: init 2023-10-06T14:26:30.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.552+0000 7f7bdd232640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:30.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.552+0000 7f7bdd232640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T14:26:30.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.552+0000 7f7bdd232640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd80b00c0 0x7f7bd813cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bdd232640 10 monclient(hunting): picked mon.noname-b con 0x7f7bd80b00c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:30.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bdd232640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bd80b3990 0x7f7bd81d99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bdd232640 10 monclient(hunting): picked mon.noname-a con 0x7f7bd80b3990 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:30.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bdd232640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7bd80b3d60 0x7f7bd8121340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bdd232640 10 monclient(hunting): picked mon.noname-c con 0x7f7bd80b3d60 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:30.560 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bdd232640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:30.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bdd232640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd81da520 con 0x7f7bd80b3990 2023-10-06T14:26:30.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bdd232640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:30.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bdd232640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd8120670 con 0x7f7bd80b00c0 2023-10-06T14:26:30.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bdd232640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:30.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bdd232640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd81d9f00 con 0x7f7bd80b3d60 2023-10-06T14:26:30.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bdd232640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:30.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bdd232640 10 monclient(hunting): authenticate will time out at 3318.739424s 2023-10-06T14:26:30.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bd759e640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7bd80b3d60 0x7f7bd8121340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bd659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bd80b3990 0x7f7bd81d99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bd6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd80b00c0 0x7f7bd813cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bd759e640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7bd80b3d60 0x7f7bd8121340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40334/0 (socket says 172.21.15.19:40334) 2023-10-06T14:26:30.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bd759e640 1 -- 172.21.15.19:0/2976208853 learned_addr learned my addr 172.21.15.19:0/2976208853 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:30.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bd759e640 10 monclient(hunting): get_auth_request con 0x7f7bd80b3d60 auth_method 0 2023-10-06T14:26:30.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bd759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:30.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bd759e640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:30.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.553+0000 7f7bd759e640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:30.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd659c640 10 monclient(hunting): get_auth_request con 0x7f7bd80b3990 auth_method 0 2023-10-06T14:26:30.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:30.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:30.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd659c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:30.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd6d9d640 10 monclient(hunting): get_auth_request con 0x7f7bd80b00c0 auth_method 0 2023-10-06T14:26:30.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:30.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd6d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:30.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd6d9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:30.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.554+0000 7f7bd659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:30.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.557+0000 7f7bd759e640 10 monclient(hunting): handle_auth_done global_id 17156 payload 824 2023-10-06T14:26:30.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.557+0000 7f7bd759e640 1 -- 172.21.15.19:0/2976208853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd80b00c0 msgr2=0x7f7bd813cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.557+0000 7f7bd759e640 1 --2- 172.21.15.19:0/2976208853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd80b00c0 0x7f7bd813cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.557+0000 7f7bd759e640 1 -- 172.21.15.19:0/2976208853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bd80b3990 msgr2=0x7f7bd81d99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.557+0000 7f7bd759e640 1 --2- 172.21.15.19:0/2976208853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bd80b3990 0x7f7bd81d99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.557+0000 7f7bd759e640 10 monclient: _finish_hunting 0 2023-10-06T14:26:30.564 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.557+0000 7f7bd759e640 1 monclient: found mon.noname-c 2023-10-06T14:26:30.565 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.557+0000 7f7bd759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:30.565 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.557+0000 7f7bd759e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T14:26:30.565 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.557+0000 7f7bd759e640 1 -- 172.21.15.19:0/2976208853 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc0009400 con 0x7f7bd80b3d60 2023-10-06T14:26:30.565 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.559+0000 7f7bd759e640 1 --2- 172.21.15.19:0/2976208853 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7bd80b3d60 0x7f7bd8121340 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800cbf0 tx=0x7f7bc800a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:30.565 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.559+0000 7f7bcf7fe640 10 client.?.objecter ms_handle_connect 0x7f7bd80b3d60 2023-10-06T14:26:30.565 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.559+0000 7f7bcf7fe640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:30.566 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bcf7fe640 1 -- 172.21.15.19:0/2976208853 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800ad40 con 0x7f7bd80b3d60 2023-10-06T14:26:30.566 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bcf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:30.566 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bcf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T14:26:30.566 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bcf7fe640 10 monclient: dump: 2023-10-06T14:26:30.566 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:30.566 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:30.567 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.567 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.567 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:30.567 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:30.567 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:30.567 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:30.567 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:30.567 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:30.567 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bcf7fe640 10 monclient: _finish_auth 0 2023-10-06T14:26:30.567 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bcf7fe640 10 monclient: _check_auth_tickets 2023-10-06T14:26:30.567 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bcf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:30.567 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bcf7fe640 1 -- 172.21.15.19:0/2976208853 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc8012400 con 0x7f7bd80b3d60 2023-10-06T14:26:30.567 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bdd232640 5 monclient: authenticate success, global_id 17156 2023-10-06T14:26:30.568 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bdd232640 10 monclient: _renew_subs 2023-10-06T14:26:30.568 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bdd232640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:30.568 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bdd232640 1 -- 172.21.15.19:0/2976208853 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd81da180 con 0x7f7bd80b3d60 2023-10-06T14:26:30.568 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bcf7fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:30.568 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bcf7fe640 1 -- 172.21.15.19:0/2976208853 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800aea0 con 0x7f7bd80b3d60 2023-10-06T14:26:30.568 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bcf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:30.568 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bcf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:30.568 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bcf7fe640 10 monclient: dump: 2023-10-06T14:26:30.568 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:30.568 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:30.568 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.568 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.569 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:30.569 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:30.569 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:30.569 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:30.569 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:30.569 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:30.569 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bd559a640 4 set_mon_vals no callback set 2023-10-06T14:26:30.569 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bdd232640 10 client.17156.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:30.569 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bdd232640 10 monclient: _renew_subs 2023-10-06T14:26:30.569 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bdd232640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:30.569 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bdd232640 1 -- 172.21.15.19:0/2976208853 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd8121b70 con 0x7f7bd80b3d60 2023-10-06T14:26:30.569 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.563+0000 7f7bdd232640 1 librados: init done 2023-10-06T14:26:30.569 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.564+0000 7f7bdd232640 10 client.17156.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:30.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.564+0000 7f7bdd232640 15 client.17156.objecter _session_command_op_assign -1 1 2023-10-06T14:26:30.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.564+0000 7f7bdd232640 20 client.17156.objecter _get_session osd=-1 returning homeless 2023-10-06T14:26:30.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.564+0000 7f7bdd232640 10 client.17156.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:30.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.564+0000 7f7bdd232640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:30.570 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.564+0000 7f7bdd232640 1 -- 172.21.15.19:0/2976208853 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7bd81da690 con 0x7f7bd80b3d60 2023-10-06T14:26:30.585 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.583+0000 7f7bcf7fe640 1 -- 172.21.15.19:0/2976208853 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f7bc8007070 con 0x7f7bd80b3d60 2023-10-06T14:26:30.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.584+0000 7f7bcf7fe640 1 --2- 172.21.15.19:0/2976208853 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f7bb806c1c0 0x7f7bb806e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.584+0000 7f7bd6d9d640 1 --2- 172.21.15.19:0/2976208853 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f7bb806c1c0 0x7f7bb806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.584+0000 7f7bd6d9d640 10 monclient: get_auth_request con 0x7f7bb806c1c0 auth_method 0 2023-10-06T14:26:30.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.584+0000 7f7bd6d9d640 1 --2- 172.21.15.19:0/2976208853 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f7bb806c1c0 0x7f7bb806e670 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0000f60 tx=0x7f7bd0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:30.587 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.584+0000 7f7bcf7fe640 10 client.17156.objecter ms_handle_connect 0x7f7bb806c1c0 2023-10-06T14:26:30.588 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 1 -- 172.21.15.19:0/2976208853 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(970..970 src has 595..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f7bc8058610 con 0x7f7bd80b3d60 2023-10-06T14:26:30.588 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 10 client.17156.objecter ms_dispatch 0x7f7bd8000b90 osd_map(970..970 src has 595..970) v4 2023-10-06T14:26:30.588 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 3 client.17156.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:30.588 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 3 client.17156.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:30.588 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 10 client.17156.objecter checking command 1 2023-10-06T14:26:30.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 1 --2- 172.21.15.19:0/2976208853 >> [v2:172.21.15.19:6808/86194,v1:172.21.15.19:6809/86194] conn(0x7f7bb80739e0 0x7f7bb8075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 20 client.17156.objecter _get_session s=0x7f7bb8073310 osd=2 3 2023-10-06T14:26:30.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 20 client.17156.objecter put_session s=0x7f7bb8073310 osd=2 3 2023-10-06T14:26:30.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 15 client.17156.objecter _session_command_op_remove -1 1 2023-10-06T14:26:30.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 20 client.17156.objecter _get_session s=0x7f7bb8073310 osd=2 3 2023-10-06T14:26:30.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 20 client.17156.objecter get_session s=0x7f7bb8073310 osd=2 3 2023-10-06T14:26:30.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 15 client.17156.objecter _session_command_op_assign 2 1 2023-10-06T14:26:30.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 20 client.17156.objecter put_session s=0x7f7bb8073310 osd=2 4 2023-10-06T14:26:30.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 10 client.17156.objecter _send_command 1 2023-10-06T14:26:30.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 1 -- 172.21.15.19:0/2976208853 --> [v2:172.21.15.19:6808/86194,v1:172.21.15.19:6809/86194] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7bb8076540 con 0x7f7bb80739e0 2023-10-06T14:26:30.589 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.586+0000 7f7bcf7fe640 20 client.17156.objecter dump_active .. 0 homeless 2023-10-06T14:26:30.590 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.587+0000 7f7bd659c640 1 --2- 172.21.15.19:0/2976208853 >> [v2:172.21.15.19:6808/86194,v1:172.21.15.19:6809/86194] conn(0x7f7bb80739e0 0x7f7bb8075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.590 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.587+0000 7f7bcf7fe640 1 -- 172.21.15.19:0/2976208853 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_get_version_reply(handle=1 version=970) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bc8054950 con 0x7f7bd80b3d60 2023-10-06T14:26:30.590 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.587+0000 7f7bcf7fe640 10 monclient: handle_get_version_reply finishing 1 version 970 2023-10-06T14:26:30.590 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.587+0000 7f7bd659c640 10 monclient: get_auth_request con 0x7f7bb80739e0 auth_method 0 2023-10-06T14:26:30.592 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.590+0000 7f7bcf7fe640 10 client.17156.objecter ms_handle_connect 0x7f7bb80739e0 2023-10-06T14:26:30.592 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.590+0000 7f7bd659c640 1 --2- 172.21.15.19:0/2976208853 >> [v2:172.21.15.19:6808/86194,v1:172.21.15.19:6809/86194] conn(0x7f7bb80739e0 0x7f7bb8075e70 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.614+0000 7f7bcf7fe640 1 -- 172.21.15.19:0/2976208853 <== osd.2 v2:172.21.15.19:6808/86194 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7bb8076540 con 0x7f7bb80739e0 2023-10-06T14:26:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.614+0000 7f7bcf7fe640 10 client.17156.objecter ms_dispatch 0x7f7bd8000b90 command_reply(tid 1: 0 ) v1 2023-10-06T14:26:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.614+0000 7f7bcf7fe640 10 client.17156.objecter _finish_command 1 = system:0 2023-10-06T14:26:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.614+0000 7f7bcf7fe640 20 client.17156.objecter put_session s=0x7f7bb8073310 osd=2 4 2023-10-06T14:26:30.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.614+0000 7f7bcf7fe640 15 client.17156.objecter _session_command_op_remove 2 1 2023-10-06T14:26:30.629 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.627+0000 7f7bdd232640 10 client.17156.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-10-06T14:26:30.629 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.627+0000 7f7bdd232640 15 client.17156.objecter _session_command_op_assign -1 2 2023-10-06T14:26:30.630 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.627+0000 7f7bdd232640 20 client.17156.objecter _get_session s=0x7f7bb8073310 osd=2 3 2023-10-06T14:26:30.630 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.627+0000 7f7bdd232640 20 client.17156.objecter put_session s=0x7f7bb8073310 osd=2 3 2023-10-06T14:26:30.630 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.627+0000 7f7bdd232640 20 client.17156.objecter _get_session s=0x7f7bb8073310 osd=2 3 2023-10-06T14:26:30.630 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.627+0000 7f7bdd232640 15 client.17156.objecter _session_command_op_remove -1 2 2023-10-06T14:26:30.630 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.627+0000 7f7bdd232640 20 client.17156.objecter get_session s=0x7f7bb8073310 osd=2 3 2023-10-06T14:26:30.630 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.627+0000 7f7bdd232640 15 client.17156.objecter _session_command_op_assign 2 2 2023-10-06T14:26:30.631 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.627+0000 7f7bdd232640 20 client.17156.objecter put_session s=0x7f7bb8073310 osd=2 4 2023-10-06T14:26:30.631 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.627+0000 7f7bdd232640 10 client.17156.objecter _send_command 2 2023-10-06T14:26:30.631 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.627+0000 7f7bdd232640 1 -- 172.21.15.19:0/2976208853 --> [v2:172.21.15.19:6808/86194,v1:172.21.15.19:6809/86194] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7bd80af3f0 con 0x7f7bb80739e0 2023-10-06T14:26:30.644 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bcf7fe640 1 -- 172.21.15.19:0/2976208853 <== osd.2 v2:172.21.15.19:6808/86194 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7bc0031b50 con 0x7f7bb80739e0 2023-10-06T14:26:30.644 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bcf7fe640 10 client.17156.objecter ms_dispatch 0x7f7bd8000b90 command_reply(tid 2: 0 ) v1 2023-10-06T14:26:30.644 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bcf7fe640 10 client.17156.objecter _finish_command 2 = system:0 2023-10-06T14:26:30.644 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bcf7fe640 20 client.17156.objecter put_session s=0x7f7bb8073310 osd=2 4 2023-10-06T14:26:30.644 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bcf7fe640 15 client.17156.objecter _session_command_op_remove 2 2 2023-10-06T14:26:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2023-10-06T14:26:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T14:26:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:} 2023-10-06T14:26:30.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 10 librados: watch_flush enter 2023-10-06T14:26:30.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 10 librados: watch_flush exit 2023-10-06T14:26:30.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 10 client.17156.objecter close_session for osd.2 2023-10-06T14:26:30.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 1 -- 172.21.15.19:0/2976208853 >> [v2:172.21.15.19:6808/86194,v1:172.21.15.19:6809/86194] conn(0x7f7bb80739e0 msgr2=0x7f7bb8075e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 1 --2- 172.21.15.19:0/2976208853 >> [v2:172.21.15.19:6808/86194,v1:172.21.15.19:6809/86194] conn(0x7f7bb80739e0 0x7f7bb8075e70 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 20 client.17156.objecter put_session s=0x7f7bb8073310 osd=2 1 2023-10-06T14:26:30.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 20 client.17156.objecter shutdown clearing up homeless session... 2023-10-06T14:26:30.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 10 client.17156.objecter successfully canceled tick 2023-10-06T14:26:30.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 1 -- 172.21.15.19:0/2976208853 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f7bb806c1c0 msgr2=0x7f7bb806e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 1 --2- 172.21.15.19:0/2976208853 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f7bb806c1c0 0x7f7bb806e670 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0000f60 tx=0x7f7bd0009040 comp rx=0 tx=0).stop 2023-10-06T14:26:30.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 10 monclient: shutdown 2023-10-06T14:26:30.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:30.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 1 -- 172.21.15.19:0/2976208853 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7bd80b3d60 msgr2=0x7f7bd8121340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 1 --2- 172.21.15.19:0/2976208853 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7bd80b3d60 0x7f7bd8121340 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800cbf0 tx=0x7f7bc800a430 comp rx=0 tx=0).stop 2023-10-06T14:26:30.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bd759e640 1 -- 172.21.15.19:0/2976208853 reap_dead start 2023-10-06T14:26:30.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 1 -- 172.21.15.19:0/2976208853 shutdown_connections 2023-10-06T14:26:30.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.642+0000 7f7bdd232640 1 -- 172.21.15.19:0/2976208853 >> 172.21.15.19:0/2976208853 conn(0x7f7bd80b6e00 msgr2=0x7f7bd813edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:30.648 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.643+0000 7f7bdd232640 1 -- 172.21.15.19:0/2976208853 shutdown_connections 2023-10-06T14:26:30.648 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.643+0000 7f7bdd232640 1 -- 172.21.15.19:0/2976208853 wait complete. 2023-10-06T14:26:30.648 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.643+0000 7f7bdd232640 1 librados: shutdown 2023-10-06T14:26:30.659 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T14:26:30.768 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.766+0000 7f5fb9320640 10 monclient: get_monmap_and_config 2023-10-06T14:26:30.768 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.766+0000 7f5fb9320640 10 monclient: build_initial_monmap 2023-10-06T14:26:30.768 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.766+0000 7f5fb9320640 10 monclient: monmap: 2023-10-06T14:26:30.768 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:30.768 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:30.769 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:30.767408+0000 2023-10-06T14:26:30.769 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:30.767408+0000 2023-10-06T14:26:30.769 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:30.769 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:30.769 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:30.769 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:30.769 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:30.769 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:30.769 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:30.769 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:30.770 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.766+0000 7f5fb9320640 1 Processor -- start 2023-10-06T14:26:30.770 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 1 -- start start 2023-10-06T14:26:30.770 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 10 monclient: init 2023-10-06T14:26:30.771 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:30.771 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2023-10-06T14:26:30.771 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5fb40b3990 0x7f5fb40af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.771 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 10 monclient(hunting): picked mon.noname-a con 0x7f5fb40b3990 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:30.771 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5fb40b3d60 addr v1:172.21.15.19:6789/0 2023-10-06T14:26:30.771 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b00c0 0x7f5fb40b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.771 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 10 monclient(hunting): picked mon.noname-b con 0x7f5fb40b00c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:30.771 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb40aebe0 con 0x7f5fb40b3d60 2023-10-06T14:26:30.771 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:30.771 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb40afd50 con 0x7f5fb40b3990 2023-10-06T14:26:30.771 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:30.771 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb4140dd0 con 0x7f5fb40b00c0 2023-10-06T14:26:30.772 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:30.772 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.767+0000 7f5fb9320640 10 monclient(hunting): authenticate will time out at 3318.954024s 2023-10-06T14:26:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.771+0000 7f5fb37fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b00c0 0x7f5fb40b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.771+0000 7f5fb37fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b00c0 0x7f5fb40b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40336/0 (socket says 172.21.15.19:40336) 2023-10-06T14:26:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.771+0000 7f5fb37fe640 1 -- 172.21.15.19:0/3194468716 learned_addr learned my addr 172.21.15.19:0/3194468716 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.771+0000 7f5fb37fe640 10 monclient(hunting): get_auth_request con 0x7f5fb40b00c0 auth_method 0 2023-10-06T14:26:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.771+0000 7f5fb37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.771+0000 7f5fb37fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.771+0000 7f5fb37fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.772+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:30.775 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.772+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:30.775 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.772+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:30.776 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.774+0000 7f5fb2ffd640 1 --2- 172.21.15.19:0/3194468716 >> v2:172.21.15.19:3300/0 conn(0x7f5fb40b3990 0x7f5fb40af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.774+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_done global_id 17159 payload 274 2023-10-06T14:26:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.774+0000 7f5fb37fe640 1 -- 172.21.15.19:0/3194468716 >> v2:172.21.15.19:3300/0 conn(0x7f5fb40b3990 msgr2=0x7f5fb40af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.774+0000 7f5fb37fe640 1 --2- 172.21.15.19:0/3194468716 >> v2:172.21.15.19:3300/0 conn(0x7f5fb40b3990 0x7f5fb40af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.774+0000 7f5fb37fe640 1 -- 172.21.15.19:0/3194468716 >> v1:172.21.15.19:6789/0 conn(0x7f5fb40b3d60 legacy=0x7f5fb4141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.774+0000 7f5fb37fe640 10 monclient: _finish_hunting 0 2023-10-06T14:26:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.774+0000 7f5fb37fe640 1 monclient: found mon.noname-b 2023-10-06T14:26:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.774+0000 7f5fb37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.774+0000 7f5fb37fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T14:26:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.774+0000 7f5fb37fe640 1 -- 172.21.15.19:0/3194468716 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb4141070 con 0x7f5fb40b00c0 2023-10-06T14:26:30.778 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.776+0000 7f5fb37fe640 1 --2- 172.21.15.19:0/3194468716 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b00c0 0x7f5fb40b0540 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009e50 tx=0x7f5fac02be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83f1a6284be2d272 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:30.786 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb1ffb640 1 -- 172.21.15.19:0/3194468716 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac02c9e0 con 0x7f5fb40b00c0 2023-10-06T14:26:30.786 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:30.786 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:30.786 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb1ffb640 10 monclient: dump: 2023-10-06T14:26:30.786 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:30.786 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:30.787 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.787 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.787 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:30.787 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:30.787 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:30.787 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:30.787 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:30.787 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:30.787 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb1ffb640 10 monclient: _finish_auth 0 2023-10-06T14:26:30.787 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb1ffb640 10 monclient: _check_auth_tickets 2023-10-06T14:26:30.787 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb1ffb640 1 -- 172.21.15.19:0/3194468716 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac02cb40 con 0x7f5fb40b00c0 2023-10-06T14:26:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb1ffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb1ffb640 1 -- 172.21.15.19:0/3194468716 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac0343f0 con 0x7f5fb40b00c0 2023-10-06T14:26:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb1ffb640 10 monclient: dump: 2023-10-06T14:26:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:30.789 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb9320640 5 monclient: authenticate success, global_id 17159 2023-10-06T14:26:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb9320640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.784+0000 7f5fb9320640 4 set_mon_vals no callback set 2023-10-06T14:26:30.793 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.785+0000 7f5fb9320640 10 monclient: shutdown 2023-10-06T14:26:30.793 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.785+0000 7f5fb9320640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:30.793 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.785+0000 7f5fb9320640 1 -- 172.21.15.19:0/3194468716 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b00c0 msgr2=0x7f5fb40b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.793 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.785+0000 7f5fb9320640 1 --2- 172.21.15.19:0/3194468716 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b00c0 0x7f5fb40b0540 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009e50 tx=0x7f5fac02be40 comp rx=0 tx=0).stop 2023-10-06T14:26:30.793 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.785+0000 7f5fb9320640 1 -- 172.21.15.19:0/3194468716 shutdown_connections 2023-10-06T14:26:30.793 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.785+0000 7f5fb9320640 1 --2- 172.21.15.19:0/3194468716 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b00c0 0x7f5fb40b0540 unknown :-1 s=CLOSED pgs=3127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.793 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.785+0000 7f5fb9320640 1 --2- 172.21.15.19:0/3194468716 >> v2:172.21.15.19:3300/0 conn(0x7f5fb40b3990 0x7f5fb40af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.793 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.785+0000 7f5fb9320640 1 -- 172.21.15.19:0/3194468716 >> 172.21.15.19:0/3194468716 conn(0x7f5fb40b6e00 msgr2=0x7f5fb40b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:30.793 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.785+0000 7f5fb9320640 1 -- 172.21.15.19:0/3194468716 shutdown_connections 2023-10-06T14:26:30.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.785+0000 7f5fb9320640 1 -- 172.21.15.19:0/3194468716 wait complete. 2023-10-06T14:26:30.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.785+0000 7f5fb9320640 10 monclient: build_initial_monmap 2023-10-06T14:26:30.794 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.785+0000 7f5fb9320640 10 monclient: monmap: 2023-10-06T14:26:30.794 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:30.794 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:30.794 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:30.794 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:30.794 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:30.794 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:30.794 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:30.794 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:30.795 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:30.795 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:30.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.786+0000 7f5fb9320640 1 librados: starting msgr at 2023-10-06T14:26:30.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.786+0000 7f5fb9320640 1 librados: starting objecter 2023-10-06T14:26:30.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.786+0000 7f5fb9320640 1 Processor -- start 2023-10-06T14:26:30.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.786+0000 7f5fb9320640 1 -- start start 2023-10-06T14:26:30.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.786+0000 7f5fb9320640 1 librados: setting wanted keys 2023-10-06T14:26:30.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.786+0000 7f5fb9320640 1 librados: calling monclient init 2023-10-06T14:26:30.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.786+0000 7f5fb9320640 10 monclient: init 2023-10-06T14:26:30.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:30.795 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T14:26:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb40b00c0 0x7f5fb41445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 10 monclient(hunting): picked mon.noname-b con 0x7f5fb40b00c0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fb40b3990 0x7f5fb413d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 10 monclient(hunting): picked mon.noname-a con 0x7f5fb40b3990 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b3d60 0x7f5fb413b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 10 monclient(hunting): picked mon.noname-c con 0x7f5fb40b3d60 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb413dc00 con 0x7f5fb40b3990 2023-10-06T14:26:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb413cee0 con 0x7f5fb40b00c0 2023-10-06T14:26:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb413d0a0 con 0x7f5fb40b3d60 2023-10-06T14:26:30.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:30.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb9320640 10 monclient(hunting): authenticate will time out at 3318.973467s 2023-10-06T14:26:30.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb37fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b3d60 0x7f5fb413b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb40b00c0 0x7f5fb41445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb37fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b3d60 0x7f5fb413b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40346/0 (socket says 172.21.15.19:40346) 2023-10-06T14:26:30.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb37fe640 1 -- 172.21.15.19:0/2947878307 learned_addr learned my addr 172.21.15.19:0/2947878307 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:30.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb37fe640 10 monclient(hunting): get_auth_request con 0x7f5fb40b3d60 auth_method 0 2023-10-06T14:26:30.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:30.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb37fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:30.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.787+0000 7f5fb37fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:30.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb2ffd640 10 monclient(hunting): get_auth_request con 0x7f5fb40b00c0 auth_method 0 2023-10-06T14:26:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb2ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb2ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb27fc640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fb40b3990 0x7f5fb413d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb27fc640 10 monclient(hunting): get_auth_request con 0x7f5fb40b3990 auth_method 0 2023-10-06T14:26:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb27fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.788+0000 7f5fb27fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.789+0000 7f5fb27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.789+0000 7f5fb27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.789+0000 7f5fb27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.791+0000 7f5fb2ffd640 10 monclient(hunting): handle_auth_done global_id 20761 payload 824 2023-10-06T14:26:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.791+0000 7f5fb2ffd640 1 -- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b3d60 msgr2=0x7f5fb413b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.791+0000 7f5fb2ffd640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b3d60 0x7f5fb413b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.791+0000 7f5fb2ffd640 1 -- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fb40b3990 msgr2=0x7f5fb413d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.791+0000 7f5fb2ffd640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fb40b3990 0x7f5fb413d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.791+0000 7f5fb2ffd640 10 monclient: _finish_hunting 0 2023-10-06T14:26:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.791+0000 7f5fb2ffd640 1 monclient: found mon.noname-b 2023-10-06T14:26:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.791+0000 7f5fb2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.791+0000 7f5fb2ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T14:26:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.791+0000 7f5fb2ffd640 1 -- 172.21.15.19:0/2947878307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac009b30 con 0x7f5fb40b00c0 2023-10-06T14:26:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.793+0000 7f5fb2ffd640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb40b00c0 0x7f5fb41445f0 secure :-1 s=READY pgs=5112 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400c5e0 tx=0x7f5fa400caa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.793+0000 7f5f9b7fe640 10 client.?.objecter ms_handle_connect 0x7f5fb40b00c0 2023-10-06T14:26:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.793+0000 7f5f9b7fe640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5fb2ffd640 1 -- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb40b00c0 msgr2=0x7f5fb41445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-10-06T14:26:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5fb2ffd640 1 -- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb40b00c0 msgr2=0x7f5fb41445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-06T14:26:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5fb2ffd640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb40b00c0 0x7f5fb41445f0 secure :-1 s=READY pgs=5112 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400c5e0 tx=0x7f5fa400caa0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T14:26:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5fb2ffd640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb40b00c0 0x7f5fb41445f0 secure :-1 s=READY pgs=5112 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400c5e0 tx=0x7f5fa400caa0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 10 monclient: ms_handle_reset current mon [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 1 -- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb40b00c0 msgr2=0x7f5fb41445f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2023-10-06T14:26:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb40b00c0 0x7f5fb41445f0 unknown :-1 s=CLOSED pgs=5112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T14:26:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fb40b3990 0x7f5fa0006ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 10 monclient(hunting): picked mon.noname-a con 0x7f5fb40b3990 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b3d60 0x7f5fa0009940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 10 monclient(hunting): picked mon.noname-c con 0x7f5fb40b3d60 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fa000a3a0 0x7f5fa000c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 10 monclient(hunting): picked mon.noname-b con 0x7f5fa000a3a0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 1 -- 172.21.15.19:0/2947878307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa000d320 con 0x7f5fb40b3990 2023-10-06T14:26:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 1 -- 172.21.15.19:0/2947878307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa000d460 con 0x7f5fa000a3a0 2023-10-06T14:26:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 1 -- 172.21.15.19:0/2947878307 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa000d620 con 0x7f5fb40b3d60 2023-10-06T14:26:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.794+0000 7f5f9b7fe640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.795+0000 7f5fb27fc640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b3d60 0x7f5fa0009940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.795+0000 7f5fb27fc640 10 monclient(hunting): get_auth_request con 0x7f5fb40b3d60 auth_method 0 2023-10-06T14:26:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.795+0000 7f5fb27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.795+0000 7f5fb27fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.795+0000 7f5fb27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T14:26:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.796+0000 7f5fb2ffd640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fb40b3990 0x7f5fa0006ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.796+0000 7f5fb27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:30.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.796+0000 7f5fb27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:30.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.796+0000 7f5fb27fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T14:26:30.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.796+0000 7f5fb37fe640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fa000a3a0 0x7f5fa000c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.796+0000 7f5fb37fe640 10 monclient(hunting): get_auth_request con 0x7f5fa000a3a0 auth_method 0 2023-10-06T14:26:30.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.796+0000 7f5fb37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:30.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.796+0000 7f5fb37fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:30.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.796+0000 7f5fb37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T14:26:30.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.796+0000 7f5fb2ffd640 10 monclient(hunting): get_auth_request con 0x7f5fb40b3990 auth_method 0 2023-10-06T14:26:30.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.796+0000 7f5fb2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:30.806 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.796+0000 7f5fb2ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:30.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.796+0000 7f5fb2ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T14:26:30.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.797+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:30.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.797+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:30.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.797+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T14:26:30.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.797+0000 7f5fb2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:30.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.797+0000 7f5fb2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:30.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.797+0000 7f5fb2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T14:26:30.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.798+0000 7f5fb27fc640 10 monclient(hunting): handle_auth_done global_id 20761 payload 293 2023-10-06T14:26:30.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.798+0000 7f5fb27fc640 1 -- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fa000a3a0 msgr2=0x7f5fa000c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.798+0000 7f5fb27fc640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fa000a3a0 0x7f5fa000c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.807 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.798+0000 7f5fb27fc640 1 -- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fb40b3990 msgr2=0x7f5fa0006ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.798+0000 7f5fb27fc640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fb40b3990 0x7f5fa0006ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.798+0000 7f5fb27fc640 10 monclient: _finish_hunting 0 2023-10-06T14:26:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.798+0000 7f5fb27fc640 1 monclient: found mon.noname-c 2023-10-06T14:26:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.798+0000 7f5fb27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.798+0000 7f5fb27fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T14:26:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.798+0000 7f5fb27fc640 1 -- 172.21.15.19:0/2947878307 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa000d8a0 con 0x7f5fb40b3d60 2023-10-06T14:26:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.799+0000 7f5fb27fc640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b3d60 0x7f5fa0009940 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4144e40 tx=0x7f5fa4008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.799+0000 7f5f9b7fe640 10 client.?.objecter ms_handle_connect 0x7f5fb40b3d60 2023-10-06T14:26:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.799+0000 7f5f9b7fe640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5f9b7fe640 1 -- 172.21.15.19:0/2947878307 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa800e780 con 0x7f5fb40b3d60 2023-10-06T14:26:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5f9b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5f9b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T14:26:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5f9b7fe640 10 monclient: dump: 2023-10-06T14:26:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:30.811 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5f9b7fe640 10 monclient: _finish_auth 0 2023-10-06T14:26:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5f9b7fe640 10 monclient: _check_auth_tickets 2023-10-06T14:26:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5f9b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5f9b7fe640 1 -- 172.21.15.19:0/2947878307 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa800e920 con 0x7f5fb40b3d60 2023-10-06T14:26:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5f9b7fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5f9b7fe640 1 -- 172.21.15.19:0/2947878307 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa800ecd0 con 0x7f5fb40b3d60 2023-10-06T14:26:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5f9b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5f9b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5f9b7fe640 10 monclient: dump: 2023-10-06T14:26:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:30.812 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5fb9320640 5 monclient: authenticate success, global_id 20761 2023-10-06T14:26:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5fb9320640 10 monclient: _renew_subs 2023-10-06T14:26:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5fb9320640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5fb9320640 1 -- 172.21.15.19:0/2947878307 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb41445f0 con 0x7f5fb40b3d60 2023-10-06T14:26:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5fb17fa640 4 set_mon_vals no callback set 2023-10-06T14:26:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5fb9320640 10 client.20761.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5fb9320640 10 monclient: _renew_subs 2023-10-06T14:26:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5fb9320640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5fb9320640 1 -- 172.21.15.19:0/2947878307 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb413d430 con 0x7f5fb40b3d60 2023-10-06T14:26:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.803+0000 7f5fb9320640 1 librados: init done 2023-10-06T14:26:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.804+0000 7f5fb9320640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.804+0000 7f5fb9320640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.804+0000 7f5fb9320640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.804+0000 7f5fb9320640 1 -- 172.21.15.19:0/2947878307 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb40a90d0 con 0x7f5fb40b3d60 2023-10-06T14:26:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.823+0000 7f5f9b7fe640 1 -- 172.21.15.19:0/2947878307 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5fa8004020 con 0x7f5fb40b3d60 2023-10-06T14:26:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.824+0000 7f5f9b7fe640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5fb40b00c0 0x7f5fa00768a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.824+0000 7f5fb2ffd640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5fb40b00c0 0x7f5fa00768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.824+0000 7f5fb2ffd640 10 monclient: get_auth_request con 0x7f5fb40b00c0 auth_method 0 2023-10-06T14:26:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.825+0000 7f5f9b7fe640 10 client.20761.objecter ms_handle_connect 0x7f5fb40b00c0 2023-10-06T14:26:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:30.825+0000 7f5fb2ffd640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5fb40b00c0 0x7f5fa00768a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400cfd0 tx=0x7f5fa401d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.076+0000 7f5f9b7fe640 1 -- 172.21.15.19:0/2947878307 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(970..970 src has 595..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f5fa8056b20 con 0x7f5fb40b3d60 2023-10-06T14:26:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.076+0000 7f5f9b7fe640 10 client.20761.objecter ms_dispatch 0x7f5fb4000b90 osd_map(970..970 src has 595..970) v4 2023-10-06T14:26:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.076+0000 7f5f9b7fe640 3 client.20761.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.076+0000 7f5f9b7fe640 3 client.20761.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.076+0000 7f5f9b7fe640 20 client.20761.objecter dump_active .. 0 homeless 2023-10-06T14:26:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.076+0000 7f5f9b7fe640 1 -- 172.21.15.19:0/2947878307 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f5fa8052e60 con 0x7f5fb40b3d60 2023-10-06T14:26:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.076+0000 7f5f9b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.076+0000 7f5f9b7fe640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:26:31.261 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.258+0000 7f5fb9320640 1 -- 172.21.15.19:0/2947878307 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fb40af110 con 0x7f5fb40b00c0 2023-10-06T14:26:31.261 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.259+0000 7f5f9b7fe640 1 -- 172.21.15.19:0/2947878307 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5fa4021040 con 0x7f5fb40b00c0 2023-10-06T14:26:31.261 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T14:26:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 10 librados: watch_flush enter 2023-10-06T14:26:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 10 librados: watch_flush exit 2023-10-06T14:26:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 20 client.20761.objecter shutdown clearing up homeless session... 2023-10-06T14:26:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 10 client.20761.objecter successfully canceled tick 2023-10-06T14:26:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 1 -- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5fb40b00c0 msgr2=0x7f5fa00768a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5fb40b00c0 0x7f5fa00768a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400cfd0 tx=0x7f5fa401d040 comp rx=0 tx=0).stop 2023-10-06T14:26:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 10 monclient: shutdown 2023-10-06T14:26:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 1 -- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b3d60 msgr2=0x7f5fa0009940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b3d60 0x7f5fa0009940 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4144e40 tx=0x7f5fa4008040 comp rx=0 tx=0).stop 2023-10-06T14:26:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 1 -- 172.21.15.19:0/2947878307 shutdown_connections 2023-10-06T14:26:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fa000a3a0 0x7f5fa000c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f5fb40b00c0 0x7f5fa00768a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f5fb40b3d60 0x7f5fa0009940 unknown :-1 s=CLOSED pgs=3128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 1 --2- 172.21.15.19:0/2947878307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fb40b3990 0x7f5fa0006ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 1 -- 172.21.15.19:0/2947878307 >> 172.21.15.19:0/2947878307 conn(0x7f5fb40b6e00 msgr2=0x7f5fb406b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 1 -- 172.21.15.19:0/2947878307 shutdown_connections 2023-10-06T14:26:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 1 -- 172.21.15.19:0/2947878307 wait complete. 2023-10-06T14:26:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.261+0000 7f5fb9320640 1 librados: shutdown 2023-10-06T14:26:31.274 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T14:26:31.275 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T14:26:31.386 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.384+0000 7f733d7ee640 10 monclient: get_monmap_and_config 2023-10-06T14:26:31.387 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.384+0000 7f733d7ee640 10 monclient: build_initial_monmap 2023-10-06T14:26:31.387 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.384+0000 7f733d7ee640 10 monclient: monmap: 2023-10-06T14:26:31.387 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:31.387 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:31.387 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:31.385747+0000 2023-10-06T14:26:31.387 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:31.385747+0000 2023-10-06T14:26:31.387 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:31.387 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:31.387 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:31.387 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:31.387 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:31.388 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:31.388 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:31.388 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:31.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 1 Processor -- start 2023-10-06T14:26:31.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 1 -- start start 2023-10-06T14:26:31.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 10 monclient: init 2023-10-06T14:26:31.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:31.390 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 10 monclient: _add_conns ranks=[4,2,3,0,1] 2023-10-06T14:26:31.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f733806b6d0 0x7f7338144200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:31.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 10 monclient(hunting): picked mon.noname-c con 0x7f733806b6d0 addr v2:172.21.15.83:3300/0 2023-10-06T14:26:31.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7338144740 0x7f733813df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:31.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 10 monclient(hunting): picked mon.noname-b con 0x7f7338144740 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:31.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f733813e460 0x7f73380b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:31.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 10 monclient(hunting): picked mon.noname-a con 0x7f733813e460 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:31.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:31.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733806baa0 con 0x7f733813e460 2023-10-06T14:26:31.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:31.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73380b2530 con 0x7f733806b6d0 2023-10-06T14:26:31.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:31.391 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73380b2810 con 0x7f7338144740 2023-10-06T14:26:31.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:31.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.385+0000 7f733d7ee640 10 monclient(hunting): authenticate will time out at 3319.572352s 2023-10-06T14:26:31.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.386+0000 7f73367fc640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7338144740 0x7f733813df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:31.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.386+0000 7f7336ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f733806b6d0 0x7f7338144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:31.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.386+0000 7f73377fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f733813e460 0x7f73380b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:31.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73367fc640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7338144740 0x7f733813df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40376/0 (socket says 172.21.15.19:40376) 2023-10-06T14:26:31.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73367fc640 1 -- 172.21.15.19:0/4039695197 learned_addr learned my addr 172.21.15.19:0/4039695197 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:31.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73367fc640 10 monclient(hunting): get_auth_request con 0x7f7338144740 auth_method 0 2023-10-06T14:26:31.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73367fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:31.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73367fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:31.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73367fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f7336ffd640 10 monclient(hunting): get_auth_request con 0x7f733806b6d0 auth_method 0 2023-10-06T14:26:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f7336ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f7336ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f7336ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73377fe640 10 monclient(hunting): get_auth_request con 0x7f733813e460 auth_method 0 2023-10-06T14:26:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73377fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73377fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f7336ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f7336ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f7336ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73367fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73367fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:31.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73367fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:31.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:31.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:31.394 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.387+0000 7f73377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.389+0000 7f7336ffd640 10 monclient(hunting): handle_auth_done global_id 20764 payload 274 2023-10-06T14:26:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.389+0000 7f7336ffd640 1 -- 172.21.15.19:0/4039695197 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7338144740 msgr2=0x7f733813df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.389+0000 7f7336ffd640 1 --2- 172.21.15.19:0/4039695197 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7338144740 0x7f733813df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.389+0000 7f7336ffd640 1 -- 172.21.15.19:0/4039695197 >> v2:172.21.15.19:3300/0 conn(0x7f733813e460 msgr2=0x7f73380b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.389+0000 7f7336ffd640 1 --2- 172.21.15.19:0/4039695197 >> v2:172.21.15.19:3300/0 conn(0x7f733813e460 0x7f73380b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.389+0000 7f7336ffd640 10 monclient: _finish_hunting 0 2023-10-06T14:26:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.389+0000 7f7336ffd640 1 monclient: found mon.noname-c 2023-10-06T14:26:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.389+0000 7f7336ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.389+0000 7f7336ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T14:26:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.389+0000 7f7336ffd640 1 -- 172.21.15.19:0/4039695197 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73380aeff0 con 0x7f733806b6d0 2023-10-06T14:26:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.390+0000 7f7336ffd640 1 --2- 172.21.15.19:0/4039695197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733806b6d0 0x7f7338144200 secure :-1 s=READY pgs=5113 cs=0 l=1 rev1=1 crypto rx=0x7f7338071cc0 tx=0x7f732c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0de6aa250df719f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:31.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f7335ffb640 1 -- 172.21.15.19:0/4039695197 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c02e070 con 0x7f733806b6d0 2023-10-06T14:26:31.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f7335ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:31.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f7335ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T14:26:31.396 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f7335ffb640 10 monclient: dump: 2023-10-06T14:26:31.396 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:31.397 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f7335ffb640 10 monclient: _finish_auth 0 2023-10-06T14:26:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f7335ffb640 10 monclient: _check_auth_tickets 2023-10-06T14:26:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f7335ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f7335ffb640 1 -- 172.21.15.19:0/4039695197 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c0315a0 con 0x7f733806b6d0 2023-10-06T14:26:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f733d7ee640 5 monclient: authenticate success, global_id 20764 2023-10-06T14:26:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f733d7ee640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f7335ffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f733d7ee640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f7335ffb640 1 -- 172.21.15.19:0/4039695197 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c031930 con 0x7f733806b6d0 2023-10-06T14:26:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f7335ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f733d7ee640 4 set_mon_vals no callback set 2023-10-06T14:26:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f7335ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.394+0000 7f7335ffb640 10 monclient: dump: 2023-10-06T14:26:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:31.399 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.395+0000 7f733d7ee640 10 monclient: shutdown 2023-10-06T14:26:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.395+0000 7f733d7ee640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.395+0000 7f733d7ee640 1 -- 172.21.15.19:0/4039695197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733806b6d0 msgr2=0x7f7338144200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.395+0000 7f733d7ee640 1 --2- 172.21.15.19:0/4039695197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733806b6d0 0x7f7338144200 secure :-1 s=READY pgs=5113 cs=0 l=1 rev1=1 crypto rx=0x7f7338071cc0 tx=0x7f732c02b840 comp rx=0 tx=0).stop 2023-10-06T14:26:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.395+0000 7f733d7ee640 1 -- 172.21.15.19:0/4039695197 shutdown_connections 2023-10-06T14:26:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.395+0000 7f733d7ee640 1 --2- 172.21.15.19:0/4039695197 >> v2:172.21.15.19:3300/0 conn(0x7f733813e460 0x7f73380b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.395+0000 7f733d7ee640 1 --2- 172.21.15.19:0/4039695197 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f7338144740 0x7f733813df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.395+0000 7f733d7ee640 1 --2- 172.21.15.19:0/4039695197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733806b6d0 0x7f7338144200 unknown :-1 s=CLOSED pgs=5113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.395+0000 7f733d7ee640 1 -- 172.21.15.19:0/4039695197 >> 172.21.15.19:0/4039695197 conn(0x7f73380665b0 msgr2=0x7f733806ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.395+0000 7f733d7ee640 1 -- 172.21.15.19:0/4039695197 shutdown_connections 2023-10-06T14:26:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.395+0000 7f733d7ee640 1 -- 172.21.15.19:0/4039695197 wait complete. 2023-10-06T14:26:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.396+0000 7f733d7ee640 10 monclient: build_initial_monmap 2023-10-06T14:26:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.396+0000 7f733d7ee640 10 monclient: monmap: 2023-10-06T14:26:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:31.404 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.396+0000 7f733d7ee640 1 librados: starting msgr at 2023-10-06T14:26:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.396+0000 7f733d7ee640 1 librados: starting objecter 2023-10-06T14:26:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.396+0000 7f733d7ee640 1 Processor -- start 2023-10-06T14:26:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.396+0000 7f733d7ee640 1 -- start start 2023-10-06T14:26:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.396+0000 7f733d7ee640 1 librados: setting wanted keys 2023-10-06T14:26:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.396+0000 7f733d7ee640 1 librados: calling monclient init 2023-10-06T14:26:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.396+0000 7f733d7ee640 10 monclient: init 2023-10-06T14:26:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T14:26:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733806b6d0 0x7f73381d8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 10 monclient(hunting): picked mon.noname-b con 0x7f733806b6d0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f733813e460 0x7f733813ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 10 monclient(hunting): picked mon.noname-c con 0x7f733813e460 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7338144740 0x7f733813b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 10 monclient(hunting): picked mon.noname-a con 0x7f7338144740 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733813b7e0 con 0x7f7338144740 2023-10-06T14:26:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73380aa7c0 con 0x7f733806b6d0 2023-10-06T14:26:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73380aa9d0 con 0x7f733813e460 2023-10-06T14:26:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f733d7ee640 10 monclient(hunting): authenticate will time out at 3319.583702s 2023-10-06T14:26:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f7336ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733806b6d0 0x7f73381d8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.397+0000 7f73367fc640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f733813e460 0x7f733813ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f73377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7338144740 0x7f733813b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f7336ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733806b6d0 0x7f73381d8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:60578/0 (socket says 172.21.15.19:60578) 2023-10-06T14:26:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f7336ffd640 1 -- 172.21.15.19:0/217733695 learned_addr learned my addr 172.21.15.19:0/217733695 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f7336ffd640 10 monclient(hunting): get_auth_request con 0x7f733806b6d0 auth_method 0 2023-10-06T14:26:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f7336ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f7336ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f7336ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f73367fc640 10 monclient(hunting): get_auth_request con 0x7f733813e460 auth_method 0 2023-10-06T14:26:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f73367fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f73367fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f73367fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f73377fe640 10 monclient(hunting): get_auth_request con 0x7f7338144740 auth_method 0 2023-10-06T14:26:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f73377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f73377fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f73377fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f7336ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f7336ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f7336ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f73367fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f73367fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.398+0000 7f73367fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.399+0000 7f73377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.399+0000 7f73377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.399+0000 7f73377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.401+0000 7f7336ffd640 10 monclient(hunting): handle_auth_done global_id 20767 payload 824 2023-10-06T14:26:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.401+0000 7f7336ffd640 1 -- 172.21.15.19:0/217733695 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f733813e460 msgr2=0x7f733813ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.401+0000 7f7336ffd640 1 --2- 172.21.15.19:0/217733695 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f733813e460 0x7f733813ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.401+0000 7f7336ffd640 1 -- 172.21.15.19:0/217733695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7338144740 msgr2=0x7f733813b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.401+0000 7f7336ffd640 1 --2- 172.21.15.19:0/217733695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7338144740 0x7f733813b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.401+0000 7f7336ffd640 10 monclient: _finish_hunting 0 2023-10-06T14:26:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.401+0000 7f7336ffd640 1 monclient: found mon.noname-b 2023-10-06T14:26:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.401+0000 7f7336ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.401+0000 7f7336ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T14:26:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.401+0000 7f7336ffd640 1 -- 172.21.15.19:0/217733695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c009400 con 0x7f733806b6d0 2023-10-06T14:26:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.403+0000 7f7336ffd640 1 --2- 172.21.15.19:0/217733695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733806b6d0 0x7f73381d8d10 secure :-1 s=READY pgs=5114 cs=0 l=1 rev1=1 crypto rx=0x7f732c000c00 tx=0x7f732c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.403+0000 7f731f7fe640 10 client.?.objecter ms_handle_connect 0x7f733806b6d0 2023-10-06T14:26:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.403+0000 7f731f7fe640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f731f7fe640 1 -- 172.21.15.19:0/217733695 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c032840 con 0x7f733806b6d0 2023-10-06T14:26:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f731f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f731f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f731f7fe640 10 monclient: dump: 2023-10-06T14:26:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:31.412 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f731f7fe640 10 monclient: _finish_auth 0 2023-10-06T14:26:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f731f7fe640 10 monclient: _check_auth_tickets 2023-10-06T14:26:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f731f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f733d7ee640 5 monclient: authenticate success, global_id 20767 2023-10-06T14:26:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f733d7ee640 10 monclient: _renew_subs 2023-10-06T14:26:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f733d7ee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f733d7ee640 1 -- 172.21.15.19:0/217733695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733813b960 con 0x7f733806b6d0 2023-10-06T14:26:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f733d7ee640 10 client.20767.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f733d7ee640 10 monclient: _renew_subs 2023-10-06T14:26:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f733d7ee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f733d7ee640 1 -- 172.21.15.19:0/217733695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73380ad420 con 0x7f733806b6d0 2023-10-06T14:26:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.405+0000 7f733d7ee640 1 librados: init done 2023-10-06T14:26:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.406+0000 7f733d7ee640 10 client.20767.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.406+0000 7f733d7ee640 15 client.20767.objecter _session_command_op_assign -1 1 2023-10-06T14:26:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.406+0000 7f733d7ee640 20 client.20767.objecter _get_session osd=-1 returning homeless 2023-10-06T14:26:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.406+0000 7f733d7ee640 10 client.20767.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.406+0000 7f733d7ee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.406+0000 7f733d7ee640 1 -- 172.21.15.19:0/217733695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f73380af260 con 0x7f733806b6d0 2023-10-06T14:26:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.406+0000 7f731f7fe640 1 -- 172.21.15.19:0/217733695 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c031f00 con 0x7f733806b6d0 2023-10-06T14:26:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.406+0000 7f731f7fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.406+0000 7f731f7fe640 1 -- 172.21.15.19:0/217733695 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c032ba0 con 0x7f733806b6d0 2023-10-06T14:26:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.406+0000 7f731f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.406+0000 7f731f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.406+0000 7f731f7fe640 10 monclient: dump: 2023-10-06T14:26:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:31.415 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.406+0000 7f73357fa640 4 set_mon_vals no callback set 2023-10-06T14:26:31.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.428+0000 7f731f7fe640 1 -- 172.21.15.19:0/217733695 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f732c032d00 con 0x7f733806b6d0 2023-10-06T14:26:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.428+0000 7f731f7fe640 1 --2- 172.21.15.19:0/217733695 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f732406c1c0 0x7f732406e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.428+0000 7f73367fc640 1 --2- 172.21.15.19:0/217733695 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f732406c1c0 0x7f732406e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:31.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.428+0000 7f73367fc640 10 monclient: get_auth_request con 0x7f732406c1c0 auth_method 0 2023-10-06T14:26:31.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.429+0000 7f73367fc640 1 --2- 172.21.15.19:0/217733695 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f732406c1c0 0x7f732406e670 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f73380b7610 tx=0x7f7328009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:31.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.429+0000 7f731f7fe640 10 client.20767.objecter ms_handle_connect 0x7f732406c1c0 2023-10-06T14:26:31.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 1 -- 172.21.15.19:0/217733695 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(970..970 src has 595..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f732c079f60 con 0x7f733806b6d0 2023-10-06T14:26:31.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 10 client.20767.objecter ms_dispatch 0x7f7338000b90 osd_map(970..970 src has 595..970) v4 2023-10-06T14:26:31.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 3 client.20767.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:31.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 3 client.20767.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:31.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 10 client.20767.objecter checking command 1 2023-10-06T14:26:31.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 1 --2- 172.21.15.19:0/217733695 >> [v2:172.21.15.19:6800/86196,v1:172.21.15.19:6801/86196] conn(0x7f73240739e0 0x7f7324075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:31.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 20 client.20767.objecter _get_session s=0x7f7324073310 osd=3 3 2023-10-06T14:26:31.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 20 client.20767.objecter put_session s=0x7f7324073310 osd=3 3 2023-10-06T14:26:31.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 15 client.20767.objecter _session_command_op_remove -1 1 2023-10-06T14:26:31.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 20 client.20767.objecter _get_session s=0x7f7324073310 osd=3 3 2023-10-06T14:26:31.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 20 client.20767.objecter get_session s=0x7f7324073310 osd=3 3 2023-10-06T14:26:31.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 15 client.20767.objecter _session_command_op_assign 3 1 2023-10-06T14:26:31.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 20 client.20767.objecter put_session s=0x7f7324073310 osd=3 4 2023-10-06T14:26:31.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 10 client.20767.objecter _send_command 1 2023-10-06T14:26:31.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 1 -- 172.21.15.19:0/217733695 --> [v2:172.21.15.19:6800/86196,v1:172.21.15.19:6801/86196] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7324076540 con 0x7f73240739e0 2023-10-06T14:26:31.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 20 client.20767.objecter dump_active .. 0 homeless 2023-10-06T14:26:31.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 1 -- 172.21.15.19:0/217733695 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=970) v2 ==== 24+0+0 (secure 0 0 0) 0x7f732c0762a0 con 0x7f733806b6d0 2023-10-06T14:26:31.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f731f7fe640 10 monclient: handle_get_version_reply finishing 1 version 970 2023-10-06T14:26:31.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.430+0000 7f73377fe640 1 --2- 172.21.15.19:0/217733695 >> [v2:172.21.15.19:6800/86196,v1:172.21.15.19:6801/86196] conn(0x7f73240739e0 0x7f7324075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:31.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.431+0000 7f73377fe640 10 monclient: get_auth_request con 0x7f73240739e0 auth_method 0 2023-10-06T14:26:31.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.433+0000 7f73377fe640 1 --2- 172.21.15.19:0/217733695 >> [v2:172.21.15.19:6800/86196,v1:172.21.15.19:6801/86196] conn(0x7f73240739e0 0x7f7324075e70 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:31.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.433+0000 7f731f7fe640 10 client.20767.objecter ms_handle_connect 0x7f73240739e0 2023-10-06T14:26:31.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.464+0000 7f731f7fe640 1 -- 172.21.15.19:0/217733695 <== osd.3 v2:172.21.15.19:6800/86196 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7324076540 con 0x7f73240739e0 2023-10-06T14:26:31.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.464+0000 7f731f7fe640 10 client.20767.objecter ms_dispatch 0x7f7338000b90 command_reply(tid 1: 0 ) v1 2023-10-06T14:26:31.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.464+0000 7f731f7fe640 10 client.20767.objecter _finish_command 1 = system:0 2023-10-06T14:26:31.466 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.464+0000 7f731f7fe640 20 client.20767.objecter put_session s=0x7f7324073310 osd=3 4 2023-10-06T14:26:31.467 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.464+0000 7f731f7fe640 15 client.20767.objecter _session_command_op_remove 3 1 2023-10-06T14:26:31.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.476+0000 7f733d7ee640 10 client.20767.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-10-06T14:26:31.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.476+0000 7f733d7ee640 15 client.20767.objecter _session_command_op_assign -1 2 2023-10-06T14:26:31.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.476+0000 7f733d7ee640 20 client.20767.objecter _get_session s=0x7f7324073310 osd=3 3 2023-10-06T14:26:31.479 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.476+0000 7f733d7ee640 20 client.20767.objecter put_session s=0x7f7324073310 osd=3 3 2023-10-06T14:26:31.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.476+0000 7f733d7ee640 20 client.20767.objecter _get_session s=0x7f7324073310 osd=3 3 2023-10-06T14:26:31.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.476+0000 7f733d7ee640 15 client.20767.objecter _session_command_op_remove -1 2 2023-10-06T14:26:31.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.476+0000 7f733d7ee640 20 client.20767.objecter get_session s=0x7f7324073310 osd=3 3 2023-10-06T14:26:31.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.476+0000 7f733d7ee640 15 client.20767.objecter _session_command_op_assign 3 2 2023-10-06T14:26:31.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.476+0000 7f733d7ee640 20 client.20767.objecter put_session s=0x7f7324073310 osd=3 4 2023-10-06T14:26:31.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.476+0000 7f733d7ee640 10 client.20767.objecter _send_command 2 2023-10-06T14:26:31.480 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.476+0000 7f733d7ee640 1 -- 172.21.15.19:0/217733695 --> [v2:172.21.15.19:6800/86196,v1:172.21.15.19:6801/86196] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7338144be0 con 0x7f73240739e0 2023-10-06T14:26:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f731f7fe640 1 -- 172.21.15.19:0/217733695 <== osd.3 v2:172.21.15.19:6800/86196 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f73300114a0 con 0x7f73240739e0 2023-10-06T14:26:31.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f731f7fe640 10 client.20767.objecter ms_dispatch 0x7f7338000b90 command_reply(tid 2: 0 ) v1 2023-10-06T14:26:31.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f731f7fe640 10 client.20767.objecter _finish_command 2 = system:0 2023-10-06T14:26:31.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f731f7fe640 20 client.20767.objecter put_session s=0x7f7324073310 osd=3 4 2023-10-06T14:26:31.494 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f731f7fe640 15 client.20767.objecter _session_command_op_remove 3 2 2023-10-06T14:26:31.494 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2023-10-06T14:26:31.494 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T14:26:31.494 INFO:teuthology.orchestra.run.smithi019.stdout:} 2023-10-06T14:26:31.496 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 10 librados: watch_flush enter 2023-10-06T14:26:31.496 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 10 librados: watch_flush exit 2023-10-06T14:26:31.496 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 10 client.20767.objecter close_session for osd.3 2023-10-06T14:26:31.496 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 1 -- 172.21.15.19:0/217733695 >> [v2:172.21.15.19:6800/86196,v1:172.21.15.19:6801/86196] conn(0x7f73240739e0 msgr2=0x7f7324075e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:31.496 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 1 --2- 172.21.15.19:0/217733695 >> [v2:172.21.15.19:6800/86196,v1:172.21.15.19:6801/86196] conn(0x7f73240739e0 0x7f7324075e70 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.496 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 20 client.20767.objecter put_session s=0x7f7324073310 osd=3 1 2023-10-06T14:26:31.496 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 20 client.20767.objecter shutdown clearing up homeless session... 2023-10-06T14:26:31.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 10 client.20767.objecter successfully canceled tick 2023-10-06T14:26:31.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 1 -- 172.21.15.19:0/217733695 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f732406c1c0 msgr2=0x7f732406e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:31.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 1 --2- 172.21.15.19:0/217733695 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f732406c1c0 0x7f732406e670 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f73380b7610 tx=0x7f7328009240 comp rx=0 tx=0).stop 2023-10-06T14:26:31.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 10 monclient: shutdown 2023-10-06T14:26:31.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:31.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 1 -- 172.21.15.19:0/217733695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733806b6d0 msgr2=0x7f73381d8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:31.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 1 --2- 172.21.15.19:0/217733695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733806b6d0 0x7f73381d8d10 secure :-1 s=READY pgs=5114 cs=0 l=1 rev1=1 crypto rx=0x7f732c000c00 tx=0x7f732c0046c0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f73377fe640 1 -- 172.21.15.19:0/217733695 reap_dead start 2023-10-06T14:26:31.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 1 -- 172.21.15.19:0/217733695 shutdown_connections 2023-10-06T14:26:31.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.491+0000 7f733d7ee640 1 -- 172.21.15.19:0/217733695 >> 172.21.15.19:0/217733695 conn(0x7f73380665b0 msgr2=0x7f73380af780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:31.497 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.492+0000 7f733d7ee640 1 -- 172.21.15.19:0/217733695 shutdown_connections 2023-10-06T14:26:31.498 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.492+0000 7f733d7ee640 1 -- 172.21.15.19:0/217733695 wait complete. 2023-10-06T14:26:31.498 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.492+0000 7f733d7ee640 1 librados: shutdown 2023-10-06T14:26:31.504 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T14:26:31.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.607+0000 7f1ca8f78640 10 monclient: get_monmap_and_config 2023-10-06T14:26:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.607+0000 7f1ca8f78640 10 monclient: build_initial_monmap 2023-10-06T14:26:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.607+0000 7f1ca8f78640 10 monclient: monmap: 2023-10-06T14:26:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:31.608859+0000 2023-10-06T14:26:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:31.608859+0000 2023-10-06T14:26:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:31.610 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:31.611 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:31.611 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:31.611 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:31.611 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:31.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.608+0000 7f1ca8f78640 1 Processor -- start 2023-10-06T14:26:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.608+0000 7f1ca8f78640 1 -- start start 2023-10-06T14:26:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.608+0000 7f1ca8f78640 10 monclient: init 2023-10-06T14:26:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca8f78640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca8f78640 10 monclient: _add_conns ranks=[1,3,4,0,2] 2023-10-06T14:26:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca8f78640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1ca406b6d0 addr v1:172.21.15.83:6789/0 2023-10-06T14:26:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca8f78640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ca4144910 0x7f1ca413df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca8f78640 10 monclient(hunting): picked mon.noname-a con 0x7f1ca4144910 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca8f78640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ca413e460 0x7f1ca414b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca8f78640 10 monclient(hunting): picked mon.noname-c con 0x7f1ca413e460 addr v2:172.21.15.83:3300/0 2023-10-06T14:26:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca8f78640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca406baa0 con 0x7f1ca406b6d0 2023-10-06T14:26:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca8f78640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca8f78640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca40b2530 con 0x7f1ca4144910 2023-10-06T14:26:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca8f78640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca8f78640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca40b2810 con 0x7f1ca413e460 2023-10-06T14:26:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca8f78640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca8f78640 10 monclient(hunting): authenticate will time out at 3319.795585s 2023-10-06T14:26:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca1d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ca4144910 0x7f1ca413df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.609+0000 7f1ca2d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ca413e460 0x7f1ca414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca2d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ca413e460 0x7f1ca414b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:60580/0 (socket says 172.21.15.19:60580) 2023-10-06T14:26:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca2d9d640 1 -- 172.21.15.19:0/2257608047 learned_addr learned my addr 172.21.15.19:0/2257608047 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca2d9d640 10 monclient(hunting): get_auth_request con 0x7f1ca413e460 auth_method 0 2023-10-06T14:26:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca2d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca2d9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca1d9b640 10 monclient(hunting): get_auth_request con 0x7f1ca4144910 auth_method 0 2023-10-06T14:26:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca1d9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca1d9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.610+0000 7f1ca2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.612+0000 7f1ca1d9b640 10 monclient(hunting): handle_auth_done global_id 15657 payload 274 2023-10-06T14:26:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.612+0000 7f1ca1d9b640 1 -- 172.21.15.19:0/2257608047 >> v2:172.21.15.83:3300/0 conn(0x7f1ca413e460 msgr2=0x7f1ca414b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.612+0000 7f1ca1d9b640 1 --2- 172.21.15.19:0/2257608047 >> v2:172.21.15.83:3300/0 conn(0x7f1ca413e460 0x7f1ca414b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.612+0000 7f1ca1d9b640 1 -- 172.21.15.19:0/2257608047 >> v1:172.21.15.83:6789/0 conn(0x7f1ca406b6d0 legacy=0x7f1ca4144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.612+0000 7f1ca1d9b640 10 monclient: _finish_hunting 0 2023-10-06T14:26:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.612+0000 7f1ca1d9b640 1 monclient: found mon.noname-a 2023-10-06T14:26:31.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.612+0000 7f1ca1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:31.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.612+0000 7f1ca1d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T14:26:31.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.612+0000 7f1ca1d9b640 1 -- 172.21.15.19:0/2257608047 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca40aeff0 con 0x7f1ca4144910 2023-10-06T14:26:31.617 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.614+0000 7f1ca1d9b640 1 --2- 172.21.15.19:0/2257608047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca4144910 0x7f1ca413df20 secure :-1 s=READY pgs=3187 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40b2280 tx=0x7f1c8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13d84f516d48067 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca159a640 1 -- 172.21.15.19:0/2257608047 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c031830 con 0x7f1ca4144910 2023-10-06T14:26:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T14:26:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca159a640 10 monclient: dump: 2023-10-06T14:26:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:31.623 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca159a640 10 monclient: _finish_auth 0 2023-10-06T14:26:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca159a640 10 monclient: _check_auth_tickets 2023-10-06T14:26:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca159a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca159a640 1 -- 172.21.15.19:0/2257608047 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8c031990 con 0x7f1ca4144910 2023-10-06T14:26:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca8f78640 5 monclient: authenticate success, global_id 15657 2023-10-06T14:26:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca8f78640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca159a640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca159a640 1 -- 172.21.15.19:0/2257608047 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c031d20 con 0x7f1ca4144910 2023-10-06T14:26:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca8f78640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca8f78640 4 set_mon_vals no callback set 2023-10-06T14:26:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T14:26:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca159a640 10 monclient: dump: 2023-10-06T14:26:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:31.625 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca8f78640 10 monclient: shutdown 2023-10-06T14:26:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca8f78640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2257608047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca4144910 msgr2=0x7f1ca413df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca8f78640 1 --2- 172.21.15.19:0/2257608047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca4144910 0x7f1ca413df20 secure :-1 s=READY pgs=3187 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40b2280 tx=0x7f1c8c02b8d0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2257608047 shutdown_connections 2023-10-06T14:26:31.626 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca8f78640 1 --2- 172.21.15.19:0/2257608047 >> v2:172.21.15.83:3300/0 conn(0x7f1ca413e460 0x7f1ca414b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.626 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca8f78640 1 --2- 172.21.15.19:0/2257608047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca4144910 0x7f1ca413df20 unknown :-1 s=CLOSED pgs=3187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.626 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2257608047 >> 172.21.15.19:0/2257608047 conn(0x7f1ca40665b0 msgr2=0x7f1ca406ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:31.626 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2257608047 shutdown_connections 2023-10-06T14:26:31.626 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.618+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2257608047 wait complete. 2023-10-06T14:26:31.626 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.619+0000 7f1ca8f78640 10 monclient: build_initial_monmap 2023-10-06T14:26:31.626 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.619+0000 7f1ca8f78640 10 monclient: monmap: 2023-10-06T14:26:31.626 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:31.626 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:31.626 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:31.626 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:31.627 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 1 librados: starting msgr at 2023-10-06T14:26:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 1 librados: starting objecter 2023-10-06T14:26:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 1 Processor -- start 2023-10-06T14:26:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 1 -- start start 2023-10-06T14:26:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 1 librados: setting wanted keys 2023-10-06T14:26:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 1 librados: calling monclient init 2023-10-06T14:26:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 10 monclient: init 2023-10-06T14:26:31.630 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:31.630 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T14:26:31.630 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca406b6d0 0x7f1ca40abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:31.630 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 10 monclient(hunting): picked mon.noname-a con 0x7f1ca406b6d0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:31.630 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca413e460 0x7f1ca41d5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 10 monclient(hunting): picked mon.noname-b con 0x7f1ca413e460 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f1ca4144910 0x7f1ca41d6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 10 monclient(hunting): picked mon.noname-c con 0x7f1ca4144910 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40ac420 con 0x7f1ca406b6d0 2023-10-06T14:26:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40ac040 con 0x7f1ca413e460 2023-10-06T14:26:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca4141ae0 con 0x7f1ca4144910 2023-10-06T14:26:31.631 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.620+0000 7f1ca8f78640 10 monclient(hunting): authenticate will time out at 3319.807180s 2023-10-06T14:26:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca2d9d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f1ca4144910 0x7f1ca41d6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca406b6d0 0x7f1ca40abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca2d9d640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f1ca4144910 0x7f1ca41d6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40388/0 (socket says 172.21.15.19:40388) 2023-10-06T14:26:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca406b6d0 0x7f1ca40abb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55398/0 (socket says 172.21.15.19:55398) 2023-10-06T14:26:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca2d9d640 1 -- 172.21.15.19:0/2550097106 learned_addr learned my addr 172.21.15.19:0/2550097106 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca2d9d640 10 monclient(hunting): get_auth_request con 0x7f1ca4144910 auth_method 0 2023-10-06T14:26:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca2d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca2d9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca259c640 10 monclient(hunting): get_auth_request con 0x7f1ca406b6d0 auth_method 0 2023-10-06T14:26:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca259c640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca259c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca1d9b640 1 --2- 172.21.15.19:0/2550097106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca413e460 0x7f1ca41d5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca1d9b640 10 monclient(hunting): get_auth_request con 0x7f1ca413e460 auth_method 0 2023-10-06T14:26:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca1d9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.621+0000 7f1ca1d9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.622+0000 7f1ca2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.622+0000 7f1ca2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.622+0000 7f1ca2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.622+0000 7f1ca259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.622+0000 7f1ca259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.622+0000 7f1ca259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.622+0000 7f1ca1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.622+0000 7f1ca1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.622+0000 7f1ca1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.624+0000 7f1ca1d9b640 10 monclient(hunting): handle_auth_done global_id 20773 payload 824 2023-10-06T14:26:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.625+0000 7f1ca1d9b640 1 -- 172.21.15.19:0/2550097106 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f1ca4144910 msgr2=0x7f1ca41d6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.625+0000 7f1ca1d9b640 1 --2- 172.21.15.19:0/2550097106 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f1ca4144910 0x7f1ca41d6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.625+0000 7f1ca1d9b640 1 -- 172.21.15.19:0/2550097106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca406b6d0 msgr2=0x7f1ca40abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.625+0000 7f1ca1d9b640 1 --2- 172.21.15.19:0/2550097106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca406b6d0 0x7f1ca40abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.625+0000 7f1ca1d9b640 10 monclient: _finish_hunting 0 2023-10-06T14:26:31.634 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.625+0000 7f1ca1d9b640 1 monclient: found mon.noname-b 2023-10-06T14:26:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.625+0000 7f1ca1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.625+0000 7f1ca1d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T14:26:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.625+0000 7f1ca1d9b640 1 -- 172.21.15.19:0/2550097106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8c009400 con 0x7f1ca413e460 2023-10-06T14:26:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.627+0000 7f1ca1d9b640 1 --2- 172.21.15.19:0/2550097106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca413e460 0x7f1ca41d5c50 secure :-1 s=READY pgs=5116 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c033000 tx=0x7f1c8c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.627+0000 7f1c9affd640 10 client.?.objecter ms_handle_connect 0x7f1ca413e460 2023-10-06T14:26:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.627+0000 7f1c9affd640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:31.636 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1c9affd640 1 -- 172.21.15.19:0/2550097106 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c03a8f0 con 0x7f1ca413e460 2023-10-06T14:26:31.636 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1c9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:31.636 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1c9affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:31.636 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1c9affd640 10 monclient: dump: 2023-10-06T14:26:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:31.637 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1c9affd640 10 monclient: _finish_auth 0 2023-10-06T14:26:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1c9affd640 10 monclient: _check_auth_tickets 2023-10-06T14:26:31.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1c9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:31.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1c9affd640 1 -- 172.21.15.19:0/2550097106 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8c03aa50 con 0x7f1ca413e460 2023-10-06T14:26:31.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1c9affd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:31.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1c9affd640 1 -- 172.21.15.19:0/2550097106 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c03ade0 con 0x7f1ca413e460 2023-10-06T14:26:31.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1ca8f78640 5 monclient: authenticate success, global_id 20773 2023-10-06T14:26:31.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1ca8f78640 10 monclient: _renew_subs 2023-10-06T14:26:31.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1ca8f78640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:31.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2550097106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca41d66d0 con 0x7f1ca413e460 2023-10-06T14:26:31.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1c9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:31.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1c9affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:31.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1c9affd640 10 monclient: dump: 2023-10-06T14:26:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:31.639 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1ca0d99640 4 set_mon_vals no callback set 2023-10-06T14:26:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1ca8f78640 10 client.20773.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1ca8f78640 10 monclient: _renew_subs 2023-10-06T14:26:31.640 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1ca8f78640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:31.640 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2550097106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca40acb40 con 0x7f1ca413e460 2023-10-06T14:26:31.640 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.630+0000 7f1ca8f78640 1 librados: init done 2023-10-06T14:26:31.640 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.631+0000 7f1ca8f78640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:31.640 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.631+0000 7f1ca8f78640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:31.640 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.631+0000 7f1ca8f78640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:31.640 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.631+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2550097106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca40ac200 con 0x7f1ca413e460 2023-10-06T14:26:31.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.871+0000 7f1c9affd640 1 -- 172.21.15.19:0/2550097106 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f1c8c030380 con 0x7f1ca413e460 2023-10-06T14:26:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.871+0000 7f1c9affd640 1 --2- 172.21.15.19:0/2550097106 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f1c8406c210 0x7f1c8406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.872+0000 7f1ca259c640 1 --2- 172.21.15.19:0/2550097106 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f1c8406c210 0x7f1c8406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.872+0000 7f1ca259c640 10 monclient: get_auth_request con 0x7f1c8406c210 auth_method 0 2023-10-06T14:26:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.872+0000 7f1ca259c640 1 --2- 172.21.15.19:0/2550097106 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f1c8406c210 0x7f1c8406e6c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c000f60 tx=0x7f1c9c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.872+0000 7f1c9affd640 10 client.20773.objecter ms_handle_connect 0x7f1c8406c210 2023-10-06T14:26:31.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.874+0000 7f1c9affd640 1 -- 172.21.15.19:0/2550097106 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(970..970 src has 595..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f1c8c07a490 con 0x7f1ca413e460 2023-10-06T14:26:31.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.874+0000 7f1c9affd640 10 client.20773.objecter ms_dispatch 0x7f1ca4000b90 osd_map(970..970 src has 595..970) v4 2023-10-06T14:26:31.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.874+0000 7f1c9affd640 3 client.20773.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:31.877 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.874+0000 7f1c9affd640 3 client.20773.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:31.877 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.874+0000 7f1c9affd640 20 client.20773.objecter dump_active .. 0 homeless 2023-10-06T14:26:31.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.884+0000 7f1c9affd640 1 -- 172.21.15.19:0/2550097106 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f1c8c0767d0 con 0x7f1ca413e460 2023-10-06T14:26:31.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.884+0000 7f1c9affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:31.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:31.884+0000 7f1c9affd640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:26:32.069 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.067+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2550097106 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ca40b1040 con 0x7f1c8406c210 2023-10-06T14:26:32.069 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.067+0000 7f1c9affd640 1 -- 172.21.15.19:0/2550097106 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1c9c0014a0 con 0x7f1c8406c210 2023-10-06T14:26:32.069 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T14:26:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.069+0000 7f1ca8f78640 10 librados: watch_flush enter 2023-10-06T14:26:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.069+0000 7f1ca8f78640 10 librados: watch_flush exit 2023-10-06T14:26:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.069+0000 7f1ca8f78640 20 client.20773.objecter shutdown clearing up homeless session... 2023-10-06T14:26:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.069+0000 7f1ca8f78640 10 client.20773.objecter successfully canceled tick 2023-10-06T14:26:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2550097106 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f1c8406c210 msgr2=0x7f1c8406e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 1 --2- 172.21.15.19:0/2550097106 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f1c8406c210 0x7f1c8406e6c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c000f60 tx=0x7f1c9c009040 comp rx=0 tx=0).stop 2023-10-06T14:26:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 10 monclient: shutdown 2023-10-06T14:26:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2550097106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca413e460 msgr2=0x7f1ca41d5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 1 --2- 172.21.15.19:0/2550097106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca413e460 0x7f1ca41d5c50 secure :-1 s=READY pgs=5116 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c033000 tx=0x7f1c8c0046c0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2550097106 shutdown_connections 2023-10-06T14:26:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 1 --2- 172.21.15.19:0/2550097106 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f1c8406c210 0x7f1c8406e6c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 1 --2- 172.21.15.19:0/2550097106 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f1ca4144910 0x7f1ca41d6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 1 --2- 172.21.15.19:0/2550097106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca413e460 0x7f1ca41d5c50 unknown :-1 s=CLOSED pgs=5116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 1 --2- 172.21.15.19:0/2550097106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca406b6d0 0x7f1ca40abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2550097106 >> 172.21.15.19:0/2550097106 conn(0x7f1ca40665b0 msgr2=0x7f1ca40b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2550097106 shutdown_connections 2023-10-06T14:26:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 1 -- 172.21.15.19:0/2550097106 wait complete. 2023-10-06T14:26:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.070+0000 7f1ca8f78640 1 librados: shutdown 2023-10-06T14:26:32.085 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T14:26:32.085 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T14:26:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.195+0000 7f8b80a3a640 10 monclient: get_monmap_and_config 2023-10-06T14:26:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.195+0000 7f8b80a3a640 10 monclient: build_initial_monmap 2023-10-06T14:26:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.195+0000 7f8b80a3a640 10 monclient: monmap: 2023-10-06T14:26:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:32.196829+0000 2023-10-06T14:26:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:32.196829+0000 2023-10-06T14:26:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:32.199 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.196+0000 7f8b80a3a640 1 Processor -- start 2023-10-06T14:26:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.196+0000 7f8b80a3a640 1 -- start start 2023-10-06T14:26:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.196+0000 7f8b80a3a640 10 monclient: init 2023-10-06T14:26:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:32.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 10 monclient: _add_conns ranks=[4,2,3,0,1] 2023-10-06T14:26:32.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8b7c06b6d0 0x7f8b7c144200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:32.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 10 monclient(hunting): picked mon.noname-c con 0x7f8b7c06b6d0 addr v2:172.21.15.83:3300/0 2023-10-06T14:26:32.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8b7c144740 0x7f8b7c13df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:32.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 10 monclient(hunting): picked mon.noname-b con 0x7f8b7c144740 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:32.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b7c13e460 0x7f8b7c0b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:32.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 10 monclient(hunting): picked mon.noname-a con 0x7f8b7c13e460 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:32.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:32.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7c06baa0 con 0x7f8b7c13e460 2023-10-06T14:26:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7c0b2530 con 0x7f8b7c06b6d0 2023-10-06T14:26:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0b2810 con 0x7f8b7c144740 2023-10-06T14:26:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b80a3a640 10 monclient(hunting): authenticate will time out at 3320.383598s 2023-10-06T14:26:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b79d9b640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8b7c144740 0x7f8b7c13df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b7ad9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b7c13e460 0x7f8b7c0b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.197+0000 7f8b7a59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8b7c06b6d0 0x7f8b7c144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b79d9b640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8b7c144740 0x7f8b7c13df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40390/0 (socket says 172.21.15.19:40390) 2023-10-06T14:26:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b79d9b640 1 -- 172.21.15.19:0/1505098779 learned_addr learned my addr 172.21.15.19:0/1505098779 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b79d9b640 10 monclient(hunting): get_auth_request con 0x7f8b7c144740 auth_method 0 2023-10-06T14:26:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b79d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b79d9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b79d9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b7ad9d640 10 monclient(hunting): get_auth_request con 0x7f8b7c13e460 auth_method 0 2023-10-06T14:26:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b7ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b7ad9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b7ad9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b7a59c640 10 monclient(hunting): get_auth_request con 0x7f8b7c06b6d0 auth_method 0 2023-10-06T14:26:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b7a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b7a59c640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b7a59c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b79d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b79d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b79d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b7ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b7ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b7ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b7a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b7a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.198+0000 7f8b7a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:32.206 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.200+0000 7f8b79d9b640 10 monclient(hunting): handle_auth_done global_id 17174 payload 274 2023-10-06T14:26:32.206 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.200+0000 7f8b79d9b640 1 -- 172.21.15.19:0/1505098779 >> v2:172.21.15.83:3300/0 conn(0x7f8b7c06b6d0 msgr2=0x7f8b7c144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:32.206 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.200+0000 7f8b79d9b640 1 --2- 172.21.15.19:0/1505098779 >> v2:172.21.15.83:3300/0 conn(0x7f8b7c06b6d0 0x7f8b7c144200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.206 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.200+0000 7f8b79d9b640 1 -- 172.21.15.19:0/1505098779 >> v2:172.21.15.19:3300/0 conn(0x7f8b7c13e460 msgr2=0x7f8b7c0b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:32.206 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.200+0000 7f8b79d9b640 1 --2- 172.21.15.19:0/1505098779 >> v2:172.21.15.19:3300/0 conn(0x7f8b7c13e460 0x7f8b7c0b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.207 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.200+0000 7f8b79d9b640 10 monclient: _finish_hunting 0 2023-10-06T14:26:32.207 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.200+0000 7f8b79d9b640 1 monclient: found mon.noname-b 2023-10-06T14:26:32.207 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.200+0000 7f8b79d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:32.207 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.200+0000 7f8b79d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T14:26:32.207 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.200+0000 7f8b79d9b640 1 -- 172.21.15.19:0/1505098779 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0aeff0 con 0x7f8b7c144740 2023-10-06T14:26:32.207 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.202+0000 7f8b79d9b640 1 --2- 172.21.15.19:0/1505098779 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8b7c144740 0x7f8b7c13df20 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c071cc0 tx=0x7f8b7002b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6309de42aec91f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b7959a640 1 -- 172.21.15.19:0/1505098779 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70031870 con 0x7f8b7c144740 2023-10-06T14:26:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b7959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b7959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b7959a640 10 monclient: dump: 2023-10-06T14:26:32.209 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:32.209 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:32.209 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.209 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.209 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:32.209 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:32.209 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:32.209 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:32.209 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:32.209 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:32.209 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b7959a640 10 monclient: _finish_auth 0 2023-10-06T14:26:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b7959a640 10 monclient: _check_auth_tickets 2023-10-06T14:26:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b7959a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b7959a640 1 -- 172.21.15.19:0/1505098779 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b700319d0 con 0x7f8b7c144740 2023-10-06T14:26:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b7959a640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b7959a640 1 -- 172.21.15.19:0/1505098779 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70031d60 con 0x7f8b7c144740 2023-10-06T14:26:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b80a3a640 5 monclient: authenticate success, global_id 17174 2023-10-06T14:26:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b80a3a640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b80a3a640 4 set_mon_vals no callback set 2023-10-06T14:26:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b7959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b7959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.205+0000 7f8b7959a640 10 monclient: dump: 2023-10-06T14:26:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:32.211 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.206+0000 7f8b80a3a640 10 monclient: shutdown 2023-10-06T14:26:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.206+0000 7f8b80a3a640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.206+0000 7f8b80a3a640 1 -- 172.21.15.19:0/1505098779 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8b7c144740 msgr2=0x7f8b7c13df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.206+0000 7f8b80a3a640 1 --2- 172.21.15.19:0/1505098779 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8b7c144740 0x7f8b7c13df20 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c071cc0 tx=0x7f8b7002b910 comp rx=0 tx=0).stop 2023-10-06T14:26:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.206+0000 7f8b80a3a640 1 -- 172.21.15.19:0/1505098779 shutdown_connections 2023-10-06T14:26:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.206+0000 7f8b80a3a640 1 --2- 172.21.15.19:0/1505098779 >> v2:172.21.15.19:3300/0 conn(0x7f8b7c13e460 0x7f8b7c0b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.206+0000 7f8b80a3a640 1 --2- 172.21.15.19:0/1505098779 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8b7c144740 0x7f8b7c13df20 unknown :-1 s=CLOSED pgs=3129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.206+0000 7f8b80a3a640 1 --2- 172.21.15.19:0/1505098779 >> v2:172.21.15.83:3300/0 conn(0x7f8b7c06b6d0 0x7f8b7c144200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.206+0000 7f8b80a3a640 1 -- 172.21.15.19:0/1505098779 >> 172.21.15.19:0/1505098779 conn(0x7f8b7c0665b0 msgr2=0x7f8b7c06ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.206+0000 7f8b80a3a640 1 -- 172.21.15.19:0/1505098779 shutdown_connections 2023-10-06T14:26:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.206+0000 7f8b80a3a640 1 -- 172.21.15.19:0/1505098779 wait complete. 2023-10-06T14:26:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.207+0000 7f8b80a3a640 10 monclient: build_initial_monmap 2023-10-06T14:26:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.207+0000 7f8b80a3a640 10 monclient: monmap: 2023-10-06T14:26:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:32.216 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.207+0000 7f8b80a3a640 1 librados: starting msgr at 2023-10-06T14:26:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.207+0000 7f8b80a3a640 1 librados: starting objecter 2023-10-06T14:26:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.207+0000 7f8b80a3a640 1 Processor -- start 2023-10-06T14:26:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.207+0000 7f8b80a3a640 1 -- start start 2023-10-06T14:26:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.207+0000 7f8b80a3a640 1 librados: setting wanted keys 2023-10-06T14:26:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.207+0000 7f8b80a3a640 1 librados: calling monclient init 2023-10-06T14:26:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.207+0000 7f8b80a3a640 10 monclient: init 2023-10-06T14:26:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 10 monclient: _add_conns ranks=[2,0,1] 2023-10-06T14:26:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8b7c06b6d0 0x7f8b7c1ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 10 monclient(hunting): picked mon.noname-c con 0x7f8b7c06b6d0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c13e460 0x7f8b7c1cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 10 monclient(hunting): picked mon.noname-a con 0x7f8b7c13e460 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b7c144740 0x7f8b7c1ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 10 monclient(hunting): picked mon.noname-b con 0x7f8b7c144740 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:32.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1cc540 con 0x7f8b7c13e460 2023-10-06T14:26:32.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:32.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1cd300 con 0x7f8b7c144740 2023-10-06T14:26:32.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:32.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c141520 con 0x7f8b7c06b6d0 2023-10-06T14:26:32.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:32.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b80a3a640 10 monclient(hunting): authenticate will time out at 3320.394823s 2023-10-06T14:26:32.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.208+0000 7f8b7a59c640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8b7c06b6d0 0x7f8b7c1ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:32.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b79d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c13e460 0x7f8b7c1cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:32.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b7c144740 0x7f8b7c1ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:32.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7a59c640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8b7c06b6d0 0x7f8b7c1ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40396/0 (socket says 172.21.15.19:40396) 2023-10-06T14:26:32.218 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7a59c640 1 -- 172.21.15.19:0/184771191 learned_addr learned my addr 172.21.15.19:0/184771191 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:32.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7a59c640 10 monclient(hunting): get_auth_request con 0x7f8b7c06b6d0 auth_method 0 2023-10-06T14:26:32.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:32.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7a59c640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:32.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7a59c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:32.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7ad9d640 10 monclient(hunting): get_auth_request con 0x7f8b7c144740 auth_method 0 2023-10-06T14:26:32.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:32.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7ad9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:32.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7ad9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:32.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b79d9b640 10 monclient(hunting): get_auth_request con 0x7f8b7c13e460 auth_method 0 2023-10-06T14:26:32.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b79d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:32.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b79d9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:32.219 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b79d9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:32.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:32.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:32.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:32.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:32.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:32.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.209+0000 7f8b7ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:32.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.210+0000 7f8b79d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:32.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.210+0000 7f8b79d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:32.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.210+0000 7f8b79d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:32.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.212+0000 7f8b7ad9d640 10 monclient(hunting): handle_auth_done global_id 20779 payload 824 2023-10-06T14:26:32.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.212+0000 7f8b7ad9d640 1 -- 172.21.15.19:0/184771191 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8b7c06b6d0 msgr2=0x7f8b7c1ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.220 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.212+0000 7f8b7ad9d640 1 --2- 172.21.15.19:0/184771191 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f8b7c06b6d0 0x7f8b7c1ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.212+0000 7f8b7ad9d640 1 -- 172.21.15.19:0/184771191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c13e460 msgr2=0x7f8b7c1cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.212+0000 7f8b7ad9d640 1 --2- 172.21.15.19:0/184771191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c13e460 0x7f8b7c1cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.212+0000 7f8b7ad9d640 10 monclient: _finish_hunting 0 2023-10-06T14:26:32.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.212+0000 7f8b7ad9d640 1 monclient: found mon.noname-b 2023-10-06T14:26:32.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.212+0000 7f8b7ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:32.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.212+0000 7f8b7ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T14:26:32.221 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.212+0000 7f8b7ad9d640 1 -- 172.21.15.19:0/184771191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b700094d0 con 0x7f8b7c144740 2023-10-06T14:26:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.214+0000 7f8b7ad9d640 1 --2- 172.21.15.19:0/184771191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b7c144740 0x7f8b7c1ccdc0 secure :-1 s=READY pgs=5117 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400efc0 tx=0x7f8b7400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.214+0000 7f8b6affd640 10 client.?.objecter ms_handle_connect 0x7f8b7c144740 2023-10-06T14:26:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.214+0000 7f8b6affd640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b6affd640 1 -- 172.21.15.19:0/184771191 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74009c00 con 0x7f8b7c144740 2023-10-06T14:26:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b6affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b6affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b6affd640 10 monclient: dump: 2023-10-06T14:26:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:32.224 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b6affd640 10 monclient: _finish_auth 0 2023-10-06T14:26:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b6affd640 10 monclient: _check_auth_tickets 2023-10-06T14:26:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b6affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b6affd640 1 -- 172.21.15.19:0/184771191 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7400f040 con 0x7f8b7c144740 2023-10-06T14:26:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b6affd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b6affd640 1 -- 172.21.15.19:0/184771191 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b740146c0 con 0x7f8b7c144740 2023-10-06T14:26:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b6affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b6affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b6affd640 10 monclient: dump: 2023-10-06T14:26:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:32.225 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b80a3a640 5 monclient: authenticate success, global_id 20779 2023-10-06T14:26:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b80a3a640 10 monclient: _renew_subs 2023-10-06T14:26:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b78d99640 4 set_mon_vals no callback set 2023-10-06T14:26:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b80a3a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b80a3a640 1 -- 172.21.15.19:0/184771191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c1c9a70 con 0x7f8b7c144740 2023-10-06T14:26:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b80a3a640 10 client.20779.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b80a3a640 10 monclient: _renew_subs 2023-10-06T14:26:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b80a3a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b80a3a640 1 -- 172.21.15.19:0/184771191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c1ca8f0 con 0x7f8b7c144740 2023-10-06T14:26:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.218+0000 7f8b80a3a640 1 librados: init done 2023-10-06T14:26:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.219+0000 7f8b80a3a640 10 client.20779.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.219+0000 7f8b80a3a640 15 client.20779.objecter _session_command_op_assign -1 1 2023-10-06T14:26:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.219+0000 7f8b80a3a640 20 client.20779.objecter _get_session osd=-1 returning homeless 2023-10-06T14:26:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.219+0000 7f8b80a3a640 10 client.20779.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.219+0000 7f8b80a3a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.219+0000 7f8b80a3a640 1 -- 172.21.15.19:0/184771191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8b7c0af260 con 0x7f8b7c144740 2023-10-06T14:26:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.236+0000 7f8b6affd640 1 -- 172.21.15.19:0/184771191 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8b74014820 con 0x7f8b7c144740 2023-10-06T14:26:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.236+0000 7f8b6affd640 1 --2- 172.21.15.19:0/184771191 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8b5c06c210 0x7f8b5c06e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.237+0000 7f8b7a59c640 1 --2- 172.21.15.19:0/184771191 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8b5c06c210 0x7f8b5c06e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.237+0000 7f8b7a59c640 10 monclient: get_auth_request con 0x7f8b5c06c210 auth_method 0 2023-10-06T14:26:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.237+0000 7f8b7a59c640 1 --2- 172.21.15.19:0/184771191 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8b5c06c210 0x7f8b5c06e6c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009b80 tx=0x7f8b6c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:32.240 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.237+0000 7f8b6affd640 10 client.20779.objecter ms_handle_connect 0x7f8b5c06c210 2023-10-06T14:26:32.241 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 1 -- 172.21.15.19:0/184771191 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(970..970 src has 595..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f8b740596c0 con 0x7f8b7c144740 2023-10-06T14:26:32.241 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 10 client.20779.objecter ms_dispatch 0x7f8b7c000b90 osd_map(970..970 src has 595..970) v4 2023-10-06T14:26:32.241 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 3 client.20779.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:32.241 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 3 client.20779.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:32.241 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 10 client.20779.objecter checking command 1 2023-10-06T14:26:32.241 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 1 --2- 172.21.15.19:0/184771191 >> [v2:172.21.15.83:6826/86123,v1:172.21.15.83:6827/86123] conn(0x7f8b5c073a30 0x7f8b5c075ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:32.241 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 20 client.20779.objecter _get_session s=0x7f8b5c073360 osd=4 3 2023-10-06T14:26:32.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 20 client.20779.objecter put_session s=0x7f8b5c073360 osd=4 3 2023-10-06T14:26:32.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 15 client.20779.objecter _session_command_op_remove -1 1 2023-10-06T14:26:32.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 20 client.20779.objecter _get_session s=0x7f8b5c073360 osd=4 3 2023-10-06T14:26:32.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 20 client.20779.objecter get_session s=0x7f8b5c073360 osd=4 3 2023-10-06T14:26:32.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 15 client.20779.objecter _session_command_op_assign 4 1 2023-10-06T14:26:32.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 20 client.20779.objecter put_session s=0x7f8b5c073360 osd=4 4 2023-10-06T14:26:32.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 10 client.20779.objecter _send_command 1 2023-10-06T14:26:32.242 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 1 -- 172.21.15.19:0/184771191 --> [v2:172.21.15.83:6826/86123,v1:172.21.15.83:6827/86123] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b5c076590 con 0x7f8b5c073a30 2023-10-06T14:26:32.243 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b6affd640 20 client.20779.objecter dump_active .. 0 homeless 2023-10-06T14:26:32.243 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.239+0000 7f8b79d9b640 1 --2- 172.21.15.19:0/184771191 >> [v2:172.21.15.83:6826/86123,v1:172.21.15.83:6827/86123] conn(0x7f8b5c073a30 0x7f8b5c075ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:32.243 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.240+0000 7f8b79d9b640 10 monclient: get_auth_request con 0x7f8b5c073a30 auth_method 0 2023-10-06T14:26:32.243 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.240+0000 7f8b6affd640 1 -- 172.21.15.19:0/184771191 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=970) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b74055a00 con 0x7f8b7c144740 2023-10-06T14:26:32.243 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.240+0000 7f8b6affd640 10 monclient: handle_get_version_reply finishing 1 version 970 2023-10-06T14:26:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.243+0000 7f8b79d9b640 1 --2- 172.21.15.19:0/184771191 >> [v2:172.21.15.83:6826/86123,v1:172.21.15.83:6827/86123] conn(0x7f8b5c073a30 0x7f8b5c075ec0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.243+0000 7f8b6affd640 10 client.20779.objecter ms_handle_connect 0x7f8b5c073a30 2023-10-06T14:26:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.264+0000 7f8b6affd640 1 -- 172.21.15.19:0/184771191 <== osd.4 v2:172.21.15.83:6826/86123 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8b5c076590 con 0x7f8b5c073a30 2023-10-06T14:26:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.264+0000 7f8b6affd640 10 client.20779.objecter ms_dispatch 0x7f8b7c000b90 command_reply(tid 1: 0 ) v1 2023-10-06T14:26:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.264+0000 7f8b6affd640 10 client.20779.objecter _finish_command 1 = system:0 2023-10-06T14:26:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.264+0000 7f8b6affd640 20 client.20779.objecter put_session s=0x7f8b5c073360 osd=4 4 2023-10-06T14:26:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.264+0000 7f8b6affd640 15 client.20779.objecter _session_command_op_remove 4 1 2023-10-06T14:26:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.277+0000 7f8b80a3a640 10 client.20779.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-10-06T14:26:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.277+0000 7f8b80a3a640 15 client.20779.objecter _session_command_op_assign -1 2 2023-10-06T14:26:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.277+0000 7f8b80a3a640 20 client.20779.objecter _get_session s=0x7f8b5c073360 osd=4 3 2023-10-06T14:26:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.277+0000 7f8b80a3a640 20 client.20779.objecter put_session s=0x7f8b5c073360 osd=4 3 2023-10-06T14:26:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.277+0000 7f8b80a3a640 20 client.20779.objecter _get_session s=0x7f8b5c073360 osd=4 3 2023-10-06T14:26:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.277+0000 7f8b80a3a640 15 client.20779.objecter _session_command_op_remove -1 2 2023-10-06T14:26:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.277+0000 7f8b80a3a640 20 client.20779.objecter get_session s=0x7f8b5c073360 osd=4 3 2023-10-06T14:26:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.277+0000 7f8b80a3a640 15 client.20779.objecter _session_command_op_assign 4 2 2023-10-06T14:26:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.277+0000 7f8b80a3a640 20 client.20779.objecter put_session s=0x7f8b5c073360 osd=4 4 2023-10-06T14:26:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.277+0000 7f8b80a3a640 10 client.20779.objecter _send_command 2 2023-10-06T14:26:32.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.277+0000 7f8b80a3a640 1 -- 172.21.15.19:0/184771191 --> [v2:172.21.15.83:6826/86123,v1:172.21.15.83:6827/86123] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8b7c144be0 con 0x7f8b5c073a30 2023-10-06T14:26:32.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b6affd640 1 -- 172.21.15.19:0/184771191 <== osd.4 v2:172.21.15.83:6826/86123 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8b70031b40 con 0x7f8b5c073a30 2023-10-06T14:26:32.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b6affd640 10 client.20779.objecter ms_dispatch 0x7f8b7c000b90 command_reply(tid 2: 0 ) v1 2023-10-06T14:26:32.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b6affd640 10 client.20779.objecter _finish_command 2 = system:0 2023-10-06T14:26:32.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b6affd640 20 client.20779.objecter put_session s=0x7f8b5c073360 osd=4 4 2023-10-06T14:26:32.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b6affd640 15 client.20779.objecter _session_command_op_remove 4 2 2023-10-06T14:26:32.294 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2023-10-06T14:26:32.295 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T14:26:32.295 INFO:teuthology.orchestra.run.smithi019.stdout:} 2023-10-06T14:26:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 10 librados: watch_flush enter 2023-10-06T14:26:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 10 librados: watch_flush exit 2023-10-06T14:26:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 10 client.20779.objecter close_session for osd.4 2023-10-06T14:26:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 1 -- 172.21.15.19:0/184771191 >> [v2:172.21.15.83:6826/86123,v1:172.21.15.83:6827/86123] conn(0x7f8b5c073a30 msgr2=0x7f8b5c075ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 1 --2- 172.21.15.19:0/184771191 >> [v2:172.21.15.83:6826/86123,v1:172.21.15.83:6827/86123] conn(0x7f8b5c073a30 0x7f8b5c075ec0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 20 client.20779.objecter put_session s=0x7f8b5c073360 osd=4 1 2023-10-06T14:26:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 20 client.20779.objecter shutdown clearing up homeless session... 2023-10-06T14:26:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 10 client.20779.objecter successfully canceled tick 2023-10-06T14:26:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 1 -- 172.21.15.19:0/184771191 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8b5c06c210 msgr2=0x7f8b5c06e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 1 --2- 172.21.15.19:0/184771191 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f8b5c06c210 0x7f8b5c06e6c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009b80 tx=0x7f8b6c009190 comp rx=0 tx=0).stop 2023-10-06T14:26:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 10 monclient: shutdown 2023-10-06T14:26:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 1 -- 172.21.15.19:0/184771191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b7c144740 msgr2=0x7f8b7c1ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 1 --2- 172.21.15.19:0/184771191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b7c144740 0x7f8b7c1ccdc0 secure :-1 s=READY pgs=5117 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400efc0 tx=0x7f8b7400c490 comp rx=0 tx=0).stop 2023-10-06T14:26:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b7ad9d640 1 -- 172.21.15.19:0/184771191 reap_dead start 2023-10-06T14:26:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 1 -- 172.21.15.19:0/184771191 shutdown_connections 2023-10-06T14:26:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 1 -- 172.21.15.19:0/184771191 >> 172.21.15.19:0/184771191 conn(0x7f8b7c0665b0 msgr2=0x7f8b7c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 1 -- 172.21.15.19:0/184771191 shutdown_connections 2023-10-06T14:26:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 1 -- 172.21.15.19:0/184771191 wait complete. 2023-10-06T14:26:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.292+0000 7f8b80a3a640 1 librados: shutdown 2023-10-06T14:26:32.305 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T14:26:32.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.409+0000 7fdff38ec640 10 monclient: get_monmap_and_config 2023-10-06T14:26:32.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.409+0000 7fdff38ec640 10 monclient: build_initial_monmap 2023-10-06T14:26:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.409+0000 7fdff38ec640 10 monclient: monmap: 2023-10-06T14:26:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:32.410772+0000 2023-10-06T14:26:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:32.410772+0000 2023-10-06T14:26:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:32.413 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:32.413 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:32.413 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:32.413 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:32.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.410+0000 7fdff38ec640 1 Processor -- start 2023-10-06T14:26:32.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.410+0000 7fdff38ec640 1 -- start start 2023-10-06T14:26:32.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.410+0000 7fdff38ec640 10 monclient: init 2023-10-06T14:26:32.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff38ec640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:32.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff38ec640 10 monclient: _add_conns ranks=[0,1,2,4,3] 2023-10-06T14:26:32.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff38ec640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdfec06d4d0 addr v1:172.21.15.19:6789/0 2023-10-06T14:26:32.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff38ec640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fdfec144270 addr v1:172.21.15.83:6789/0 2023-10-06T14:26:32.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff38ec640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdfec06ac10 0x7fdfec14c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:32.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff38ec640 10 monclient(hunting): picked mon.noname-b con 0x7fdfec06ac10 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:32.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff38ec640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec06d8a0 con 0x7fdfec06d4d0 2023-10-06T14:26:32.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff38ec640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec0aeb30 con 0x7fdfec144270 2023-10-06T14:26:32.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff38ec640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:32.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff38ec640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec071300 con 0x7fdfec06ac10 2023-10-06T14:26:32.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff38ec640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:32.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff38ec640 10 monclient(hunting): authenticate will time out at 3320.597637s 2023-10-06T14:26:32.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff1e89640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdfec06ac10 0x7fdfec14c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:32.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff1688640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdfec06d4d0 0x7fdfec0ae420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48064/0 (socket says 172.21.15.19:48064) 2023-10-06T14:26:32.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.411+0000 7fdff1688640 1 -- 172.21.15.19:0/811678448 learned_addr learned my addr 172.21.15.19:0/811678448 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T14:26:32.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.412+0000 7fdff1e89640 10 monclient(hunting): get_auth_request con 0x7fdfec06ac10 auth_method 0 2023-10-06T14:26:32.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.412+0000 7fdff1e89640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.412+0000 7fdff1e89640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.412+0000 7fdff1e89640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.412+0000 7fdff1e89640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.412+0000 7fdff1e89640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.412+0000 7fdff1e89640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.414+0000 7fdff1e89640 10 monclient(hunting): handle_auth_done global_id 17180 payload 274 2023-10-06T14:26:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.414+0000 7fdff1e89640 1 -- 172.21.15.19:0/811678448 >> v1:172.21.15.83:6789/0 conn(0x7fdfec144270 legacy=0x7fdfec06a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.414+0000 7fdff1e89640 1 -- 172.21.15.19:0/811678448 >> v1:172.21.15.19:6789/0 conn(0x7fdfec06d4d0 legacy=0x7fdfec0ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.414+0000 7fdff1e89640 10 monclient: _finish_hunting 0 2023-10-06T14:26:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.414+0000 7fdff1e89640 1 monclient: found mon.noname-b 2023-10-06T14:26:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.414+0000 7fdfdffff640 1 -- 172.21.15.19:0/811678448 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181628715 0 0) 0x7fdfec06d8a0 con 0x7fdfec06d4d0 2023-10-06T14:26:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.414+0000 7fdff1e89640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.414+0000 7fdff1e89640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T14:26:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.414+0000 7fdff1e89640 1 -- 172.21.15.19:0/811678448 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec0aef80 con 0x7fdfec06ac10 2023-10-06T14:26:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.414+0000 7fdfdffff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2023-10-06T14:26:32.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.416+0000 7fdff1e89640 1 --2- 172.21.15.19:0/811678448 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdfec06ac10 0x7fdfec14c5b0 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009dc0 tx=0x7fdfe402be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d9274c23e1710c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:32.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdfdffff640 1 -- 172.21.15.19:0/811678448 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe402cc40 con 0x7fdfec06ac10 2023-10-06T14:26:32.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdfdffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:32.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdfdffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:32.422 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdfdffff640 10 monclient: dump: 2023-10-06T14:26:32.422 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:32.422 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:32.422 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.422 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.422 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:32.423 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdfdffff640 10 monclient: _finish_auth 0 2023-10-06T14:26:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdfdffff640 10 monclient: _check_auth_tickets 2023-10-06T14:26:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdfdffff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdfdffff640 1 -- 172.21.15.19:0/811678448 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe402cda0 con 0x7fdfec06ac10 2023-10-06T14:26:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdff38ec640 5 monclient: authenticate success, global_id 17180 2023-10-06T14:26:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdff38ec640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdfdffff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdff38ec640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:32.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdfdffff640 1 -- 172.21.15.19:0/811678448 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4034660 con 0x7fdfec06ac10 2023-10-06T14:26:32.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdff38ec640 4 set_mon_vals no callback set 2023-10-06T14:26:32.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdfdffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:32.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdfdffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:32.424 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdfdffff640 10 monclient: dump: 2023-10-06T14:26:32.424 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:32.424 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:32.424 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.424 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.424 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:32.424 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:32.424 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:32.425 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:32.425 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:32.425 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:32.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdff38ec640 10 monclient: shutdown 2023-10-06T14:26:32.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdff38ec640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:32.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdff38ec640 1 -- 172.21.15.19:0/811678448 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdfec06ac10 msgr2=0x7fdfec14c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdff38ec640 1 --2- 172.21.15.19:0/811678448 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdfec06ac10 0x7fdfec14c5b0 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009dc0 tx=0x7fdfe402be90 comp rx=0 tx=0).stop 2023-10-06T14:26:32.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdff38ec640 1 -- 172.21.15.19:0/811678448 shutdown_connections 2023-10-06T14:26:32.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdff38ec640 1 --2- 172.21.15.19:0/811678448 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdfec06ac10 0x7fdfec14c5b0 unknown :-1 s=CLOSED pgs=3130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdff38ec640 1 -- 172.21.15.19:0/811678448 >> 172.21.15.19:0/811678448 conn(0x7fdfec0b4d50 msgr2=0x7fdfec0b5120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:32.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.420+0000 7fdff38ec640 1 -- 172.21.15.19:0/811678448 shutdown_connections 2023-10-06T14:26:32.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.421+0000 7fdff38ec640 1 -- 172.21.15.19:0/811678448 wait complete. 2023-10-06T14:26:32.428 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.421+0000 7fdff38ec640 10 monclient: build_initial_monmap 2023-10-06T14:26:32.429 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.421+0000 7fdff38ec640 10 monclient: monmap: 2023-10-06T14:26:32.429 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:32.429 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:32.429 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:32.429 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:32.429 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:32.429 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:32.429 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:32.429 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:32.429 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:32.429 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:32.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 1 librados: starting msgr at 2023-10-06T14:26:32.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 1 librados: starting objecter 2023-10-06T14:26:32.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 1 Processor -- start 2023-10-06T14:26:32.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 1 -- start start 2023-10-06T14:26:32.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 1 librados: setting wanted keys 2023-10-06T14:26:32.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 1 librados: calling monclient init 2023-10-06T14:26:32.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 10 monclient: init 2023-10-06T14:26:32.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:32.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T14:26:32.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfec06d4d0 0x7fdfec1d9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:32.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 10 monclient(hunting): picked mon.noname-b con 0x7fdfec06d4d0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:32.430 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfec144270 0x7fdfec139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:32.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 10 monclient(hunting): picked mon.noname-a con 0x7fdfec144270 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:32.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdfec1d9ba0 0x7fdfec139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:32.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 10 monclient(hunting): picked mon.noname-c con 0x7fdfec1d9ba0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:32.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:32.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec13fe60 con 0x7fdfec144270 2023-10-06T14:26:32.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:32.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec13f0f0 con 0x7fdfec06d4d0 2023-10-06T14:26:32.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:32.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec13f260 con 0x7fdfec1d9ba0 2023-10-06T14:26:32.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:32.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.422+0000 7fdff38ec640 10 monclient(hunting): authenticate will time out at 3320.609233s 2023-10-06T14:26:32.431 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff1e89640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdfec1d9ba0 0x7fdfec139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:32.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff1688640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfec06d4d0 0x7fdfec1d9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:32.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff0e87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfec144270 0x7fdfec139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:32.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff1e89640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdfec1d9ba0 0x7fdfec139e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40422/0 (socket says 172.21.15.19:40422) 2023-10-06T14:26:32.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff1e89640 1 -- 172.21.15.19:0/3572878723 learned_addr learned my addr 172.21.15.19:0/3572878723 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:32.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff1e89640 10 monclient(hunting): get_auth_request con 0x7fdfec1d9ba0 auth_method 0 2023-10-06T14:26:32.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff1e89640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:32.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff1e89640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:32.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff1e89640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:32.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff1688640 10 monclient(hunting): get_auth_request con 0x7fdfec06d4d0 auth_method 0 2023-10-06T14:26:32.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff1688640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:32.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff1688640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:32.432 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff1688640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:32.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff0e87640 10 monclient(hunting): get_auth_request con 0x7fdfec144270 auth_method 0 2023-10-06T14:26:32.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff0e87640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:32.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff0e87640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:32.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.423+0000 7fdff0e87640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:32.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.424+0000 7fdff1e89640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:32.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.424+0000 7fdff1e89640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:32.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.424+0000 7fdff1e89640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:32.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.424+0000 7fdff1688640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:32.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.424+0000 7fdff1688640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:32.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.424+0000 7fdff1688640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:32.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.424+0000 7fdff0e87640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:32.433 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.424+0000 7fdff0e87640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:32.434 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.424+0000 7fdff0e87640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:32.435 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.426+0000 7fdff1e89640 10 monclient(hunting): handle_auth_done global_id 17183 payload 824 2023-10-06T14:26:32.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.426+0000 7fdff1e89640 1 -- 172.21.15.19:0/3572878723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfec06d4d0 msgr2=0x7fdfec1d9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.426+0000 7fdff1e89640 1 --2- 172.21.15.19:0/3572878723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfec06d4d0 0x7fdfec1d9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.426+0000 7fdff1e89640 1 -- 172.21.15.19:0/3572878723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfec144270 msgr2=0x7fdfec139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.426+0000 7fdff1e89640 1 --2- 172.21.15.19:0/3572878723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfec144270 0x7fdfec139940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.426+0000 7fdff1e89640 10 monclient: _finish_hunting 0 2023-10-06T14:26:32.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.426+0000 7fdff1e89640 1 monclient: found mon.noname-c 2023-10-06T14:26:32.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.426+0000 7fdff1e89640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:32.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.426+0000 7fdff1e89640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T14:26:32.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.426+0000 7fdff1e89640 1 -- 172.21.15.19:0/3572878723 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe4009a70 con 0x7fdfec1d9ba0 2023-10-06T14:26:32.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.426+0000 7fdff1688640 1 --2- 172.21.15.19:0/3572878723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfec06d4d0 0x7fdfec1d9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T14:26:32.436 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.429+0000 7fdff1e89640 1 --2- 172.21.15.19:0/3572878723 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdfec1d9ba0 0x7fdfec139e80 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4037040 tx=0x7fdfe4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:32.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.429+0000 7fdfddffb640 10 client.?.objecter ms_handle_connect 0x7fdfec1d9ba0 2023-10-06T14:26:32.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.429+0000 7fdfddffb640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:32.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdfddffb640 1 -- 172.21.15.19:0/3572878723 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe403e610 con 0x7fdfec1d9ba0 2023-10-06T14:26:32.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdfddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:32.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdfddffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T14:26:32.437 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdfddffb640 10 monclient: dump: 2023-10-06T14:26:32.437 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:32.437 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:32.437 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.437 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.437 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:32.437 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:32.438 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:32.438 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:32.438 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:32.438 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:32.438 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdfddffb640 10 monclient: _finish_auth 0 2023-10-06T14:26:32.438 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdfddffb640 10 monclient: _check_auth_tickets 2023-10-06T14:26:32.438 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdfddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:32.438 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdfddffb640 1 -- 172.21.15.19:0/3572878723 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe403e770 con 0x7fdfec1d9ba0 2023-10-06T14:26:32.438 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdfddffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:32.438 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdfddffb640 1 -- 172.21.15.19:0/3572878723 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe403eb00 con 0x7fdfec1d9ba0 2023-10-06T14:26:32.438 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdff38ec640 5 monclient: authenticate success, global_id 17183 2023-10-06T14:26:32.438 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdff38ec640 10 monclient: _renew_subs 2023-10-06T14:26:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdff38ec640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdff38ec640 1 -- 172.21.15.19:0/3572878723 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfec13a4e0 con 0x7fdfec1d9ba0 2023-10-06T14:26:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdfddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdfddffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdfddffb640 10 monclient: dump: 2023-10-06T14:26:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:32.440 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:32.440 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:32.440 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:32.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.432+0000 7fdfdeffd640 4 set_mon_vals no callback set 2023-10-06T14:26:32.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.433+0000 7fdff38ec640 10 client.17183.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:32.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.433+0000 7fdff38ec640 10 monclient: _renew_subs 2023-10-06T14:26:32.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.433+0000 7fdff38ec640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:32.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.433+0000 7fdff38ec640 1 -- 172.21.15.19:0/3572878723 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfec140560 con 0x7fdfec1d9ba0 2023-10-06T14:26:32.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.433+0000 7fdff38ec640 1 librados: init done 2023-10-06T14:26:32.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.433+0000 7fdff38ec640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:32.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.433+0000 7fdff38ec640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:32.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.433+0000 7fdff38ec640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:32.441 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.433+0000 7fdff38ec640 1 -- 172.21.15.19:0/3572878723 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfec1400a0 con 0x7fdfec1d9ba0 2023-10-06T14:26:32.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.453+0000 7fdfddffb640 1 -- 172.21.15.19:0/3572878723 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fdfe403b050 con 0x7fdfec1d9ba0 2023-10-06T14:26:32.455 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.453+0000 7fdfddffb640 1 --2- 172.21.15.19:0/3572878723 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fdfd006c210 0x7fdfd006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.454+0000 7fdff1688640 1 --2- 172.21.15.19:0/3572878723 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fdfd006c210 0x7fdfd006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.454+0000 7fdff1688640 10 monclient: get_auth_request con 0x7fdfd006c210 auth_method 0 2023-10-06T14:26:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.454+0000 7fdff1688640 1 --2- 172.21.15.19:0/3572878723 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fdfd006c210 0x7fdfd006e6c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8007f70 tx=0x7fdfe8007f00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.454+0000 7fdfddffb640 10 client.17183.objecter ms_handle_connect 0x7fdfd006c210 2023-10-06T14:26:32.707 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.705+0000 7fdfddffb640 1 -- 172.21.15.19:0/3572878723 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(970..970 src has 595..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fdfe407acd0 con 0x7fdfec1d9ba0 2023-10-06T14:26:32.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.705+0000 7fdfddffb640 10 client.17183.objecter ms_dispatch 0x7fdfec000b90 osd_map(970..970 src has 595..970) v4 2023-10-06T14:26:32.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.705+0000 7fdfddffb640 3 client.17183.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:32.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.705+0000 7fdfddffb640 3 client.17183.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:32.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.705+0000 7fdfddffb640 20 client.17183.objecter dump_active .. 0 homeless 2023-10-06T14:26:32.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.705+0000 7fdfddffb640 1 -- 172.21.15.19:0/3572878723 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7fdfe4077010 con 0x7fdfec1d9ba0 2023-10-06T14:26:32.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.705+0000 7fdfddffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:32.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.705+0000 7fdfddffb640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:26:32.888 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.886+0000 7fdff38ec640 1 -- 172.21.15.19:0/3572878723 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfec06bb70 con 0x7fdfd006c210 2023-10-06T14:26:32.888 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.886+0000 7fdfddffb640 1 -- 172.21.15.19:0/3572878723 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdfe8001cf0 con 0x7fdfd006c210 2023-10-06T14:26:32.889 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T14:26:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 10 librados: watch_flush enter 2023-10-06T14:26:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 10 librados: watch_flush exit 2023-10-06T14:26:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 20 client.17183.objecter shutdown clearing up homeless session... 2023-10-06T14:26:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 10 client.17183.objecter successfully canceled tick 2023-10-06T14:26:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 1 -- 172.21.15.19:0/3572878723 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fdfd006c210 msgr2=0x7fdfd006e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 1 --2- 172.21.15.19:0/3572878723 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fdfd006c210 0x7fdfd006e6c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8007f70 tx=0x7fdfe8007f00 comp rx=0 tx=0).stop 2023-10-06T14:26:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 10 monclient: shutdown 2023-10-06T14:26:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 1 -- 172.21.15.19:0/3572878723 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdfec1d9ba0 msgr2=0x7fdfec139e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 1 --2- 172.21.15.19:0/3572878723 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdfec1d9ba0 0x7fdfec139e80 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4037040 tx=0x7fdfe4004a40 comp rx=0 tx=0).stop 2023-10-06T14:26:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 1 -- 172.21.15.19:0/3572878723 shutdown_connections 2023-10-06T14:26:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 1 --2- 172.21.15.19:0/3572878723 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fdfd006c210 0x7fdfd006e6c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 1 --2- 172.21.15.19:0/3572878723 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fdfec1d9ba0 0x7fdfec139e80 unknown :-1 s=CLOSED pgs=3131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 1 --2- 172.21.15.19:0/3572878723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfec144270 0x7fdfec139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 1 --2- 172.21.15.19:0/3572878723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfec06d4d0 0x7fdfec1d9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 1 -- 172.21.15.19:0/3572878723 >> 172.21.15.19:0/3572878723 conn(0x7fdfec0b4d50 msgr2=0x7fdfec0ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 1 -- 172.21.15.19:0/3572878723 shutdown_connections 2023-10-06T14:26:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 1 -- 172.21.15.19:0/3572878723 wait complete. 2023-10-06T14:26:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:32.889+0000 7fdff38ec640 1 librados: shutdown 2023-10-06T14:26:32.903 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T14:26:32.904 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T14:26:33.016 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.014+0000 7fb85c02f640 10 monclient: get_monmap_and_config 2023-10-06T14:26:33.016 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.014+0000 7fb85c02f640 10 monclient: build_initial_monmap 2023-10-06T14:26:33.016 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.014+0000 7fb85c02f640 10 monclient: monmap: 2023-10-06T14:26:33.016 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:33.016 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:33.017 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:33.015356+0000 2023-10-06T14:26:33.017 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:33.015356+0000 2023-10-06T14:26:33.017 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:33.017 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.017 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:33.017 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:33.017 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:33.017 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:33.017 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:33.017 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.018 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.014+0000 7fb85c02f640 1 Processor -- start 2023-10-06T14:26:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.015+0000 7fb85c02f640 1 -- start start 2023-10-06T14:26:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.015+0000 7fb85c02f640 10 monclient: init 2023-10-06T14:26:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.015+0000 7fb85c02f640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.015+0000 7fb85c02f640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2023-10-06T14:26:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.015+0000 7fb85c02f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb85406b6d0 0x7fb854144200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.015+0000 7fb85c02f640 10 monclient(hunting): picked mon.noname-a con 0x7fb85406b6d0 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.015+0000 7fb85c02f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb854144740 addr v1:172.21.15.83:6789/0 2023-10-06T14:26:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.015+0000 7fb85c02f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb85413e630 addr v1:172.21.15.19:6789/0 2023-10-06T14:26:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.015+0000 7fb85c02f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb85406baa0 con 0x7fb85413e630 2023-10-06T14:26:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.015+0000 7fb85c02f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8540b2530 con 0x7fb854144740 2023-10-06T14:26:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.015+0000 7fb85c02f640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.020 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.015+0000 7fb85c02f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8540b2810 con 0x7fb85406b6d0 2023-10-06T14:26:33.020 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.015+0000 7fb85c02f640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:33.020 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.015+0000 7fb85c02f640 10 monclient(hunting): authenticate will time out at 3321.202146s 2023-10-06T14:26:33.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.023+0000 7fb8595ca640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb854144740 0x7fb85413df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:43008/0 (socket says 172.21.15.19:43008) 2023-10-06T14:26:33.026 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.023+0000 7fb8595ca640 1 -- 172.21.15.19:0/3372686450 learned_addr learned my addr 172.21.15.19:0/3372686450 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T14:26:33.031 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.029+0000 7fb859dcb640 1 --2- 172.21.15.19:0/3372686450 >> v2:172.21.15.19:3300/0 conn(0x7fb85406b6d0 0x7fb854144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.030+0000 7fb859dcb640 10 monclient(hunting): get_auth_request con 0x7fb85406b6d0 auth_method 0 2023-10-06T14:26:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.030+0000 7fb859dcb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.030+0000 7fb859dcb640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.030+0000 7fb859dcb640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.034 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.032+0000 7fb858dc9640 1 -- 172.21.15.19:0/3372686450 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368838802 0 0) 0x7fb8540b2530 con 0x7fb854144740 2023-10-06T14:26:33.034 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.032+0000 7fb858dc9640 10 monclient(hunting): _negotiate 2023-10-06T14:26:33.035 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.032+0000 7fb858dc9640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.035 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.032+0000 7fb858dc9640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.035 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.032+0000 7fb858dc9640 10 monclient(hunting): my global_id is 20788 2023-10-06T14:26:33.035 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.032+0000 7fb858dc9640 1 -- 172.21.15.19:0/3372686450 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb840003520 con 0x7fb854144740 2023-10-06T14:26:33.035 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.032+0000 7fb859dcb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:33.035 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.032+0000 7fb859dcb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:33.035 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.032+0000 7fb859dcb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.035+0000 7fb859dcb640 10 monclient(hunting): handle_auth_done global_id 15675 payload 274 2023-10-06T14:26:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.035+0000 7fb859dcb640 1 -- 172.21.15.19:0/3372686450 >> v1:172.21.15.83:6789/0 conn(0x7fb854144740 legacy=0x7fb85413df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.035+0000 7fb859dcb640 1 -- 172.21.15.19:0/3372686450 >> v1:172.21.15.19:6789/0 conn(0x7fb85413e630 legacy=0x7fb85414b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:33.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.035+0000 7fb859dcb640 10 monclient: _finish_hunting 0 2023-10-06T14:26:33.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.035+0000 7fb859dcb640 1 monclient: found mon.noname-a 2023-10-06T14:26:33.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.035+0000 7fb859dcb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:33.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.035+0000 7fb859dcb640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T14:26:33.038 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.035+0000 7fb859dcb640 1 -- 172.21.15.19:0/3372686450 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8540aeff0 con 0x7fb85406b6d0 2023-10-06T14:26:33.040 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.038+0000 7fb859dcb640 1 --2- 172.21.15.19:0/3372686450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb85406b6d0 0x7fb854144200 secure :-1 s=READY pgs=3190 cs=0 l=1 rev1=1 crypto rx=0x7fb8500097a0 tx=0x7fb85002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d11ac58f268eaf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb858dc9640 1 -- 172.21.15.19:0/3372686450 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850031830 con 0x7fb85406b6d0 2023-10-06T14:26:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb858dc9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb858dc9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T14:26:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb858dc9640 10 monclient: dump: 2023-10-06T14:26:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:33.052 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb858dc9640 10 monclient: _finish_auth 0 2023-10-06T14:26:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb858dc9640 10 monclient: _check_auth_tickets 2023-10-06T14:26:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb858dc9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:33.053 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb858dc9640 1 -- 172.21.15.19:0/3372686450 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb850031990 con 0x7fb85406b6d0 2023-10-06T14:26:33.053 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb85c02f640 5 monclient: authenticate success, global_id 15675 2023-10-06T14:26:33.053 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb85c02f640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:33.053 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb858dc9640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:33.053 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb85c02f640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:33.053 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb85c02f640 4 set_mon_vals no callback set 2023-10-06T14:26:33.053 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb858dc9640 1 -- 172.21.15.19:0/3372686450 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850031d20 con 0x7fb85406b6d0 2023-10-06T14:26:33.053 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb858dc9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:33.053 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb858dc9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T14:26:33.053 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.044+0000 7fb858dc9640 10 monclient: dump: 2023-10-06T14:26:33.053 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:33.053 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:33.054 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.045+0000 7fb85c02f640 10 monclient: shutdown 2023-10-06T14:26:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.045+0000 7fb85c02f640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.045+0000 7fb85c02f640 1 -- 172.21.15.19:0/3372686450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb85406b6d0 msgr2=0x7fb854144200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.045+0000 7fb85c02f640 1 --2- 172.21.15.19:0/3372686450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb85406b6d0 0x7fb854144200 secure :-1 s=READY pgs=3190 cs=0 l=1 rev1=1 crypto rx=0x7fb8500097a0 tx=0x7fb85002b8d0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.045+0000 7fb85c02f640 1 -- 172.21.15.19:0/3372686450 shutdown_connections 2023-10-06T14:26:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.045+0000 7fb85c02f640 1 --2- 172.21.15.19:0/3372686450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb85406b6d0 0x7fb854144200 unknown :-1 s=CLOSED pgs=3190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.045+0000 7fb85c02f640 1 -- 172.21.15.19:0/3372686450 >> 172.21.15.19:0/3372686450 conn(0x7fb8540665b0 msgr2=0x7fb85406ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.045+0000 7fb85c02f640 1 -- 172.21.15.19:0/3372686450 shutdown_connections 2023-10-06T14:26:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.045+0000 7fb85c02f640 1 -- 172.21.15.19:0/3372686450 wait complete. 2023-10-06T14:26:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.046+0000 7fb85c02f640 10 monclient: build_initial_monmap 2023-10-06T14:26:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.046+0000 7fb85c02f640 10 monclient: monmap: 2023-10-06T14:26:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:33.056 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.046+0000 7fb85c02f640 1 librados: starting msgr at 2023-10-06T14:26:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.046+0000 7fb85c02f640 1 librados: starting objecter 2023-10-06T14:26:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.046+0000 7fb85c02f640 1 Processor -- start 2023-10-06T14:26:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 1 -- start start 2023-10-06T14:26:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 1 librados: setting wanted keys 2023-10-06T14:26:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 1 librados: calling monclient init 2023-10-06T14:26:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 10 monclient: init 2023-10-06T14:26:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 10 monclient: _add_conns ranks=[0,1,2] 2023-10-06T14:26:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb85406b6d0 0x7fb854142120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 10 monclient(hunting): picked mon.noname-a con 0x7fb85406b6d0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb85413e630 0x7fb85413ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 10 monclient(hunting): picked mon.noname-b con 0x7fb85413e630 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb854144740 0x7fb85413b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 10 monclient(hunting): picked mon.noname-c con 0x7fb854144740 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85413ba10 con 0x7fb85406b6d0 2023-10-06T14:26:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb854141bf0 con 0x7fb85413e630 2023-10-06T14:26:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb854141d30 con 0x7fb854144740 2023-10-06T14:26:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb85c02f640 10 monclient(hunting): authenticate will time out at 3321.233923s 2023-10-06T14:26:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.047+0000 7fb859dcb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb85406b6d0 0x7fb854142120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb85a5cc640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb854144740 0x7fb85413b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb8595ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb85413e630 0x7fb85413ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb85a5cc640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb854144740 0x7fb85413b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40428/0 (socket says 172.21.15.19:40428) 2023-10-06T14:26:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb85a5cc640 1 -- 172.21.15.19:0/341492001 learned_addr learned my addr 172.21.15.19:0/341492001 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb85a5cc640 10 monclient(hunting): get_auth_request con 0x7fb854144740 auth_method 0 2023-10-06T14:26:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb85a5cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:33.059 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb85a5cc640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb85a5cc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb859dcb640 10 monclient(hunting): get_auth_request con 0x7fb85406b6d0 auth_method 0 2023-10-06T14:26:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb859dcb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb859dcb640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb859dcb640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb8595ca640 10 monclient(hunting): get_auth_request con 0x7fb85413e630 auth_method 0 2023-10-06T14:26:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb8595ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb8595ca640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb8595ca640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb8595ca640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb8595ca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb8595ca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb859dcb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb859dcb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb859dcb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb85a5cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb85a5cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.048+0000 7fb85a5cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.051+0000 7fb8595ca640 10 monclient(hunting): handle_auth_done global_id 20791 payload 824 2023-10-06T14:26:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.051+0000 7fb8595ca640 1 -- 172.21.15.19:0/341492001 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb854144740 msgr2=0x7fb85413b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.051+0000 7fb8595ca640 1 --2- 172.21.15.19:0/341492001 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fb854144740 0x7fb85413b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.051+0000 7fb8595ca640 1 -- 172.21.15.19:0/341492001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb85406b6d0 msgr2=0x7fb854142120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.051+0000 7fb8595ca640 1 --2- 172.21.15.19:0/341492001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb85406b6d0 0x7fb854142120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.051+0000 7fb8595ca640 10 monclient: _finish_hunting 0 2023-10-06T14:26:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.051+0000 7fb8595ca640 1 monclient: found mon.noname-b 2023-10-06T14:26:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.051+0000 7fb8595ca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.051+0000 7fb8595ca640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T14:26:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.051+0000 7fb8595ca640 1 -- 172.21.15.19:0/341492001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb850009400 con 0x7fb85413e630 2023-10-06T14:26:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.053+0000 7fb8595ca640 1 --2- 172.21.15.19:0/341492001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb85413e630 0x7fb85413ad60 secure :-1 s=READY pgs=5120 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c390 tx=0x7fb844001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.053+0000 7fb84a7fc640 10 client.?.objecter ms_handle_connect 0x7fb85413e630 2023-10-06T14:26:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.053+0000 7fb84a7fc640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.057+0000 7fb84a7fc640 1 -- 172.21.15.19:0/341492001 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844005ce0 con 0x7fb85413e630 2023-10-06T14:26:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.057+0000 7fb84a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.057+0000 7fb84a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.057+0000 7fb84a7fc640 10 monclient: dump: 2023-10-06T14:26:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:33.065 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.057+0000 7fb84a7fc640 10 monclient: _finish_auth 0 2023-10-06T14:26:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.057+0000 7fb84a7fc640 10 monclient: _check_auth_tickets 2023-10-06T14:26:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.057+0000 7fb84a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.057+0000 7fb84a7fc640 1 -- 172.21.15.19:0/341492001 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb844005e40 con 0x7fb85413e630 2023-10-06T14:26:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.057+0000 7fb85c02f640 5 monclient: authenticate success, global_id 20791 2023-10-06T14:26:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.057+0000 7fb84a7fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.057+0000 7fb85c02f640 10 monclient: _renew_subs 2023-10-06T14:26:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.057+0000 7fb84a7fc640 1 -- 172.21.15.19:0/341492001 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8440072f0 con 0x7fb85413e630 2023-10-06T14:26:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb85c02f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb85c02f640 1 -- 172.21.15.19:0/341492001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb854140a90 con 0x7fb85413e630 2023-10-06T14:26:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb84a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb84a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb84a7fc640 10 monclient: dump: 2023-10-06T14:26:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:33.067 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.057+0000 7fb84bfff640 4 set_mon_vals no callback set 2023-10-06T14:26:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb85c02f640 10 client.20791.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb85c02f640 10 monclient: _renew_subs 2023-10-06T14:26:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb85c02f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb85c02f640 1 -- 172.21.15.19:0/341492001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb854144330 con 0x7fb85413e630 2023-10-06T14:26:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb85c02f640 1 librados: init done 2023-10-06T14:26:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb85c02f640 10 client.20791.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb85c02f640 15 client.20791.objecter _session_command_op_assign -1 1 2023-10-06T14:26:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb85c02f640 20 client.20791.objecter _get_session osd=-1 returning homeless 2023-10-06T14:26:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb85c02f640 10 client.20791.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb85c02f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.058+0000 7fb85c02f640 1 -- 172.21.15.19:0/341492001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb854144b60 con 0x7fb85413e630 2023-10-06T14:26:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.076+0000 7fb84a7fc640 1 -- 172.21.15.19:0/341492001 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb844007510 con 0x7fb85413e630 2023-10-06T14:26:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.077+0000 7fb84a7fc640 1 --2- 172.21.15.19:0/341492001 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fb83806c210 0x7fb83806e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.077+0000 7fb859dcb640 1 --2- 172.21.15.19:0/341492001 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fb83806c210 0x7fb83806e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.077+0000 7fb859dcb640 10 monclient: get_auth_request con 0x7fb83806c210 auth_method 0 2023-10-06T14:26:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.077+0000 7fb84a7fc640 10 client.20791.objecter ms_handle_connect 0x7fb83806c210 2023-10-06T14:26:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.077+0000 7fb859dcb640 1 --2- 172.21.15.19:0/341492001 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fb83806c210 0x7fb83806e6c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb850036000 tx=0x7fb85003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 1 -- 172.21.15.19:0/341492001 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(970..970 src has 595..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fb844007e20 con 0x7fb85413e630 2023-10-06T14:26:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 10 client.20791.objecter ms_dispatch 0x7fb854000b90 osd_map(970..970 src has 595..970) v4 2023-10-06T14:26:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 3 client.20791.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 3 client.20791.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 10 client.20791.objecter checking command 1 2023-10-06T14:26:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 1 --2- 172.21.15.19:0/341492001 >> [v2:172.21.15.83:6818/86126,v1:172.21.15.83:6819/86126] conn(0x7fb838073a30 0x7fb838075ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 20 client.20791.objecter _get_session s=0x7fb838073360 osd=5 3 2023-10-06T14:26:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 20 client.20791.objecter put_session s=0x7fb838073360 osd=5 3 2023-10-06T14:26:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 15 client.20791.objecter _session_command_op_remove -1 1 2023-10-06T14:26:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 20 client.20791.objecter _get_session s=0x7fb838073360 osd=5 3 2023-10-06T14:26:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 20 client.20791.objecter get_session s=0x7fb838073360 osd=5 3 2023-10-06T14:26:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 15 client.20791.objecter _session_command_op_assign 5 1 2023-10-06T14:26:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 20 client.20791.objecter put_session s=0x7fb838073360 osd=5 4 2023-10-06T14:26:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 10 client.20791.objecter _send_command 1 2023-10-06T14:26:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 1 -- 172.21.15.19:0/341492001 --> [v2:172.21.15.83:6818/86126,v1:172.21.15.83:6819/86126] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb838076590 con 0x7fb838073a30 2023-10-06T14:26:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.079+0000 7fb84a7fc640 20 client.20791.objecter dump_active .. 0 homeless 2023-10-06T14:26:33.083 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.080+0000 7fb85a5cc640 1 --2- 172.21.15.19:0/341492001 >> [v2:172.21.15.83:6818/86126,v1:172.21.15.83:6819/86126] conn(0x7fb838073a30 0x7fb838075ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.083 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.080+0000 7fb85a5cc640 10 monclient: get_auth_request con 0x7fb838073a30 auth_method 0 2023-10-06T14:26:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.082+0000 7fb84a7fc640 1 -- 172.21.15.19:0/341492001 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=970) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb844053e60 con 0x7fb85413e630 2023-10-06T14:26:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.082+0000 7fb84a7fc640 10 monclient: handle_get_version_reply finishing 1 version 970 2023-10-06T14:26:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.083+0000 7fb85a5cc640 1 --2- 172.21.15.19:0/341492001 >> [v2:172.21.15.83:6818/86126,v1:172.21.15.83:6819/86126] conn(0x7fb838073a30 0x7fb838075ec0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.083+0000 7fb84a7fc640 10 client.20791.objecter ms_handle_connect 0x7fb838073a30 2023-10-06T14:26:33.109 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.107+0000 7fb84a7fc640 1 -- 172.21.15.19:0/341492001 <== osd.5 v2:172.21.15.83:6818/86126 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb838076590 con 0x7fb838073a30 2023-10-06T14:26:33.109 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.107+0000 7fb84a7fc640 10 client.20791.objecter ms_dispatch 0x7fb854000b90 command_reply(tid 1: 0 ) v1 2023-10-06T14:26:33.109 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.107+0000 7fb84a7fc640 10 client.20791.objecter _finish_command 1 = system:0 2023-10-06T14:26:33.109 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.107+0000 7fb84a7fc640 20 client.20791.objecter put_session s=0x7fb838073360 osd=5 4 2023-10-06T14:26:33.109 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.107+0000 7fb84a7fc640 15 client.20791.objecter _session_command_op_remove 5 1 2023-10-06T14:26:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.119+0000 7fb85c02f640 10 client.20791.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-10-06T14:26:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.119+0000 7fb85c02f640 15 client.20791.objecter _session_command_op_assign -1 2 2023-10-06T14:26:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.119+0000 7fb85c02f640 20 client.20791.objecter _get_session s=0x7fb838073360 osd=5 3 2023-10-06T14:26:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.119+0000 7fb85c02f640 20 client.20791.objecter put_session s=0x7fb838073360 osd=5 3 2023-10-06T14:26:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.119+0000 7fb85c02f640 20 client.20791.objecter _get_session s=0x7fb838073360 osd=5 3 2023-10-06T14:26:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.119+0000 7fb85c02f640 15 client.20791.objecter _session_command_op_remove -1 2 2023-10-06T14:26:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.119+0000 7fb85c02f640 20 client.20791.objecter get_session s=0x7fb838073360 osd=5 3 2023-10-06T14:26:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.119+0000 7fb85c02f640 15 client.20791.objecter _session_command_op_assign 5 2 2023-10-06T14:26:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.119+0000 7fb85c02f640 20 client.20791.objecter put_session s=0x7fb838073360 osd=5 4 2023-10-06T14:26:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.119+0000 7fb85c02f640 10 client.20791.objecter _send_command 2 2023-10-06T14:26:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.119+0000 7fb85c02f640 1 -- 172.21.15.19:0/341492001 --> [v2:172.21.15.83:6818/86126,v1:172.21.15.83:6819/86126] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb8540af260 con 0x7fb838073a30 2023-10-06T14:26:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb84a7fc640 1 -- 172.21.15.19:0/341492001 <== osd.5 v2:172.21.15.83:6818/86126 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb84c005320 con 0x7fb838073a30 2023-10-06T14:26:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb84a7fc640 10 client.20791.objecter ms_dispatch 0x7fb854000b90 command_reply(tid 2: 0 ) v1 2023-10-06T14:26:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb84a7fc640 10 client.20791.objecter _finish_command 2 = system:0 2023-10-06T14:26:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb84a7fc640 20 client.20791.objecter put_session s=0x7fb838073360 osd=5 4 2023-10-06T14:26:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb84a7fc640 15 client.20791.objecter _session_command_op_remove 5 2 2023-10-06T14:26:33.136 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2023-10-06T14:26:33.137 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T14:26:33.137 INFO:teuthology.orchestra.run.smithi019.stdout:} 2023-10-06T14:26:33.138 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 10 librados: watch_flush enter 2023-10-06T14:26:33.138 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 10 librados: watch_flush exit 2023-10-06T14:26:33.138 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 10 client.20791.objecter close_session for osd.5 2023-10-06T14:26:33.138 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 1 -- 172.21.15.19:0/341492001 >> [v2:172.21.15.83:6818/86126,v1:172.21.15.83:6819/86126] conn(0x7fb838073a30 msgr2=0x7fb838075ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.138 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 1 --2- 172.21.15.19:0/341492001 >> [v2:172.21.15.83:6818/86126,v1:172.21.15.83:6819/86126] conn(0x7fb838073a30 0x7fb838075ec0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.138 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 20 client.20791.objecter put_session s=0x7fb838073360 osd=5 1 2023-10-06T14:26:33.138 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 20 client.20791.objecter shutdown clearing up homeless session... 2023-10-06T14:26:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 10 client.20791.objecter successfully canceled tick 2023-10-06T14:26:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 1 -- 172.21.15.19:0/341492001 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fb83806c210 msgr2=0x7fb83806e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 1 --2- 172.21.15.19:0/341492001 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fb83806c210 0x7fb83806e6c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb850036000 tx=0x7fb85003c040 comp rx=0 tx=0).stop 2023-10-06T14:26:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 10 monclient: shutdown 2023-10-06T14:26:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 1 -- 172.21.15.19:0/341492001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb85413e630 msgr2=0x7fb85413ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 1 --2- 172.21.15.19:0/341492001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb85413e630 0x7fb85413ad60 secure :-1 s=READY pgs=5120 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c390 tx=0x7fb844001e70 comp rx=0 tx=0).stop 2023-10-06T14:26:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85a5cc640 1 -- 172.21.15.19:0/341492001 reap_dead start 2023-10-06T14:26:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 1 -- 172.21.15.19:0/341492001 shutdown_connections 2023-10-06T14:26:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 1 -- 172.21.15.19:0/341492001 >> 172.21.15.19:0/341492001 conn(0x7fb8540665b0 msgr2=0x7fb8540af740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 1 -- 172.21.15.19:0/341492001 shutdown_connections 2023-10-06T14:26:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 1 -- 172.21.15.19:0/341492001 wait complete. 2023-10-06T14:26:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.134+0000 7fb85c02f640 1 librados: shutdown 2023-10-06T14:26:33.146 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T14:26:33.257 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.255+0000 7f3d7d15c640 10 monclient: get_monmap_and_config 2023-10-06T14:26:33.257 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.255+0000 7f3d7d15c640 10 monclient: build_initial_monmap 2023-10-06T14:26:33.257 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.255+0000 7f3d7d15c640 10 monclient: monmap: 2023-10-06T14:26:33.257 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:33.257 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:33.257 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:33.256106+0000 2023-10-06T14:26:33.258 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:33.256106+0000 2023-10-06T14:26:33.258 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:33.258 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.258 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:33.258 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:33.258 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:33.258 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:33.258 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:33.258 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.255+0000 7f3d7d15c640 1 Processor -- start 2023-10-06T14:26:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.255+0000 7f3d7d15c640 1 -- start start 2023-10-06T14:26:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.255+0000 7f3d7d15c640 10 monclient: init 2023-10-06T14:26:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.256+0000 7f3d7d15c640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.256+0000 7f3d7d15c640 10 monclient: _add_conns ranks=[0,1,4,3,2] 2023-10-06T14:26:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.256+0000 7f3d7d15c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3d780b3990 addr v1:172.21.15.19:6789/0 2023-10-06T14:26:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.256+0000 7f3d7d15c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3d780b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T14:26:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.256+0000 7f3d7d15c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3d780b07d0 0x7f3d780af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.256+0000 7f3d7d15c640 10 monclient(hunting): picked mon.noname-c con 0x7f3d780b07d0 addr v2:172.21.15.83:3300/0 2023-10-06T14:26:33.261 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.256+0000 7f3d7d15c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d780aebe0 con 0x7f3d780b3990 2023-10-06T14:26:33.262 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.256+0000 7f3d7d15c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d78141070 con 0x7f3d780b3d60 2023-10-06T14:26:33.262 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.256+0000 7f3d7d15c640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.262 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.256+0000 7f3d7d15c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d78140dd0 con 0x7f3d780b07d0 2023-10-06T14:26:33.262 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.256+0000 7f3d7d15c640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:33.262 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.256+0000 7f3d7d15c640 10 monclient(hunting): authenticate will time out at 3321.442711s 2023-10-06T14:26:33.263 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.256+0000 7f3d76d9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3d780b3990 0x7f3d78141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48082/0 (socket says 172.21.15.19:48082) 2023-10-06T14:26:33.263 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.256+0000 7f3d76d9d640 1 -- 172.21.15.19:0/1378396157 learned_addr learned my addr 172.21.15.19:0/1378396157 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T14:26:33.263 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.257+0000 7f3d7759e640 1 --2- 172.21.15.19:0/1378396157 >> v2:172.21.15.83:3300/0 conn(0x7f3d780b07d0 0x7f3d780af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.263 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.257+0000 7f3d7759e640 10 monclient(hunting): get_auth_request con 0x7f3d780b07d0 auth_method 0 2023-10-06T14:26:33.263 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.257+0000 7f3d7759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:33.263 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.257+0000 7f3d7759e640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.263 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.257+0000 7f3d7759e640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.264 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.258+0000 7f3d7759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.258+0000 7f3d7759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.258+0000 7f3d7759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.259+0000 7f3d75d9b640 1 -- 172.21.15.19:0/1378396157 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604808982 0 0) 0x7f3d780aebe0 con 0x7f3d780b3990 2023-10-06T14:26:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.259+0000 7f3d75d9b640 10 monclient(hunting): _negotiate 2023-10-06T14:26:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.259+0000 7f3d75d9b640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.259+0000 7f3d75d9b640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.259+0000 7f3d75d9b640 10 monclient(hunting): my global_id is 15681 2023-10-06T14:26:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.259+0000 7f3d75d9b640 1 -- 172.21.15.19:0/1378396157 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d60003520 con 0x7f3d780b3990 2023-10-06T14:26:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.260+0000 7f3d7759e640 10 monclient(hunting): handle_auth_done global_id 20794 payload 274 2023-10-06T14:26:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.260+0000 7f3d7759e640 1 -- 172.21.15.19:0/1378396157 >> v1:172.21.15.83:6789/0 conn(0x7f3d780b3d60 legacy=0x7f3d780b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.260+0000 7f3d7759e640 1 -- 172.21.15.19:0/1378396157 >> v1:172.21.15.19:6789/0 conn(0x7f3d780b3990 legacy=0x7f3d78141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.266 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.260+0000 7f3d7759e640 10 monclient: _finish_hunting 0 2023-10-06T14:26:33.266 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.260+0000 7f3d7759e640 1 monclient: found mon.noname-c 2023-10-06T14:26:33.266 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.260+0000 7f3d7759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:33.266 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.260+0000 7f3d7759e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T14:26:33.266 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.260+0000 7f3d7759e640 1 -- 172.21.15.19:0/1378396157 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d78141c10 con 0x7f3d780b07d0 2023-10-06T14:26:33.266 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.262+0000 7f3d7759e640 1 --2- 172.21.15.19:0/1378396157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d780b07d0 0x7f3d780af810 secure :-1 s=READY pgs=5122 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009d90 tx=0x7f3d6802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fa496083e5de27 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:33.270 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d75d9b640 1 -- 172.21.15.19:0/1378396157 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6802cc60 con 0x7f3d780b07d0 2023-10-06T14:26:33.270 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d75d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:33.270 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d75d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T14:26:33.270 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d75d9b640 10 monclient: dump: 2023-10-06T14:26:33.271 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:33.271 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:33.271 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.271 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.271 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:33.271 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.271 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:33.271 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:33.271 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:33.271 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.271 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d75d9b640 10 monclient: _finish_auth 0 2023-10-06T14:26:33.271 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d75d9b640 10 monclient: _check_auth_tickets 2023-10-06T14:26:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d75d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d75d9b640 1 -- 172.21.15.19:0/1378396157 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6802cdc0 con 0x7f3d780b07d0 2023-10-06T14:26:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d7d15c640 5 monclient: authenticate success, global_id 20794 2023-10-06T14:26:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d7d15c640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d75d9b640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d7d15c640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d75d9b640 1 -- 172.21.15.19:0/1378396157 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d68034660 con 0x7f3d780b07d0 2023-10-06T14:26:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d75d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d7d15c640 4 set_mon_vals no callback set 2023-10-06T14:26:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d75d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.268+0000 7f3d75d9b640 10 monclient: dump: 2023-10-06T14:26:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:33.273 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:33.273 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.273 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.273 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:33.273 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.273 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:33.273 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:33.273 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:33.273 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.276 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.269+0000 7f3d7d15c640 10 monclient: shutdown 2023-10-06T14:26:33.276 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.269+0000 7f3d7d15c640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:33.276 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.269+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1378396157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d780b07d0 msgr2=0x7f3d780af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.277 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.269+0000 7f3d7d15c640 1 --2- 172.21.15.19:0/1378396157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d780b07d0 0x7f3d780af810 secure :-1 s=READY pgs=5122 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009d90 tx=0x7f3d6802beb0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.277 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.269+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1378396157 shutdown_connections 2023-10-06T14:26:33.277 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.269+0000 7f3d7d15c640 1 --2- 172.21.15.19:0/1378396157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d780b07d0 0x7f3d780af810 unknown :-1 s=CLOSED pgs=5122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.277 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.269+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1378396157 >> 172.21.15.19:0/1378396157 conn(0x7f3d780b6e00 msgr2=0x7f3d780b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:33.277 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.269+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1378396157 shutdown_connections 2023-10-06T14:26:33.277 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.269+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1378396157 wait complete. 2023-10-06T14:26:33.277 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.270+0000 7f3d7d15c640 10 monclient: build_initial_monmap 2023-10-06T14:26:33.277 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.270+0000 7f3d7d15c640 10 monclient: monmap: 2023-10-06T14:26:33.277 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:33.277 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:33.277 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:33.277 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:33.278 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:33.278 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.278 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:33.278 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:33.278 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:33.278 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.270+0000 7f3d7d15c640 1 librados: starting msgr at 2023-10-06T14:26:33.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.270+0000 7f3d7d15c640 1 librados: starting objecter 2023-10-06T14:26:33.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.270+0000 7f3d7d15c640 1 Processor -- start 2023-10-06T14:26:33.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.270+0000 7f3d7d15c640 1 -- start start 2023-10-06T14:26:33.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.270+0000 7f3d7d15c640 1 librados: setting wanted keys 2023-10-06T14:26:33.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.270+0000 7f3d7d15c640 1 librados: calling monclient init 2023-10-06T14:26:33.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.270+0000 7f3d7d15c640 10 monclient: init 2023-10-06T14:26:33.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:33.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T14:26:33.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d780b07d0 0x7f3d7813d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 10 monclient(hunting): picked mon.noname-b con 0x7f3d780b07d0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:33.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3d780b3990 0x7f3d7813abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 10 monclient(hunting): picked mon.noname-c con 0x7f3d780b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:33.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d780b3d60 0x7f3d781d9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 10 monclient(hunting): picked mon.noname-a con 0x7f3d780b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:33.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.281 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d78137cf0 con 0x7f3d780b3d60 2023-10-06T14:26:33.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780a8490 con 0x7f3d780b07d0 2023-10-06T14:26:33.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780a8770 con 0x7f3d780b3990 2023-10-06T14:26:33.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:33.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7d15c640 10 monclient(hunting): authenticate will time out at 3321.457545s 2023-10-06T14:26:33.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d780b3d60 0x7f3d781d9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d7659c640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3d780b3990 0x7f3d7813abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.271+0000 7f3d76d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d780b07d0 0x7f3d7813d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7659c640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3d780b3990 0x7f3d7813abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40442/0 (socket says 172.21.15.19:40442) 2023-10-06T14:26:33.282 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7659c640 1 -- 172.21.15.19:0/1597526183 learned_addr learned my addr 172.21.15.19:0/1597526183 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7659c640 10 monclient(hunting): get_auth_request con 0x7f3d780b3990 auth_method 0 2023-10-06T14:26:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7659c640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7659c640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d76d9d640 10 monclient(hunting): get_auth_request con 0x7f3d780b07d0 auth_method 0 2023-10-06T14:26:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d76d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d76d9d640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d76d9d640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7759e640 10 monclient(hunting): get_auth_request con 0x7f3d780b3d60 auth_method 0 2023-10-06T14:26:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7759e640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7759e640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d76d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d76d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d76d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.272+0000 7f3d7759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.275+0000 7f3d76d9d640 10 monclient(hunting): handle_auth_done global_id 20800 payload 824 2023-10-06T14:26:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.275+0000 7f3d76d9d640 1 -- 172.21.15.19:0/1597526183 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3d780b3990 msgr2=0x7f3d7813abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.275+0000 7f3d76d9d640 1 --2- 172.21.15.19:0/1597526183 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3d780b3990 0x7f3d7813abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.275+0000 7f3d76d9d640 1 -- 172.21.15.19:0/1597526183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d780b3d60 msgr2=0x7f3d781d9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.275+0000 7f3d76d9d640 1 --2- 172.21.15.19:0/1597526183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d780b3d60 0x7f3d781d9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.275+0000 7f3d76d9d640 10 monclient: _finish_hunting 0 2023-10-06T14:26:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.275+0000 7f3d76d9d640 1 monclient: found mon.noname-b 2023-10-06T14:26:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.275+0000 7f3d76d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.275+0000 7f3d76d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T14:26:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.275+0000 7f3d76d9d640 1 -- 172.21.15.19:0/1597526183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d68009a70 con 0x7f3d780b07d0 2023-10-06T14:26:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.277+0000 7f3d76d9d640 1 --2- 172.21.15.19:0/1597526183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d780b07d0 0x7f3d7813d930 secure :-1 s=READY pgs=5123 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000c460 tx=0x7f3d7000c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.277+0000 7f3d5f7fe640 10 client.?.objecter ms_handle_connect 0x7f3d780b07d0 2023-10-06T14:26:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.277+0000 7f3d5f7fe640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d5f7fe640 1 -- 172.21.15.19:0/1597526183 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70005ce0 con 0x7f3d780b07d0 2023-10-06T14:26:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d5f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d5f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d5f7fe640 10 monclient: dump: 2023-10-06T14:26:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:33.288 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d5f7fe640 10 monclient: _finish_auth 0 2023-10-06T14:26:33.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d5f7fe640 10 monclient: _check_auth_tickets 2023-10-06T14:26:33.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d5f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:33.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d5f7fe640 1 -- 172.21.15.19:0/1597526183 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d70005e40 con 0x7f3d780b07d0 2023-10-06T14:26:33.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d5f7fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:33.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d5f7fe640 1 -- 172.21.15.19:0/1597526183 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d700072b0 con 0x7f3d780b07d0 2023-10-06T14:26:33.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d5f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:33.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d5f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:33.289 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d5f7fe640 10 monclient: dump: 2023-10-06T14:26:33.289 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:33.289 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:33.289 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:33.290 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d7d15c640 5 monclient: authenticate success, global_id 20800 2023-10-06T14:26:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.281+0000 7f3d7d15c640 10 monclient: _renew_subs 2023-10-06T14:26:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.281+0000 7f3d7d15c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.281+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1597526183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d780a9cb0 con 0x7f3d780b07d0 2023-10-06T14:26:33.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.280+0000 7f3d7559a640 4 set_mon_vals no callback set 2023-10-06T14:26:33.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.281+0000 7f3d7d15c640 10 client.20800.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:33.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.281+0000 7f3d7d15c640 10 monclient: _renew_subs 2023-10-06T14:26:33.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.281+0000 7f3d7d15c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:33.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.281+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1597526183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7813ca80 con 0x7f3d780b07d0 2023-10-06T14:26:33.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.281+0000 7f3d7d15c640 1 librados: init done 2023-10-06T14:26:33.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.281+0000 7f3d7d15c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:33.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.281+0000 7f3d7d15c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:33.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.281+0000 7f3d7d15c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:33.291 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.281+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1597526183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d780678e0 con 0x7f3d780b07d0 2023-10-06T14:26:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.300+0000 7f3d5f7fe640 1 -- 172.21.15.19:0/1597526183 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f3d70007410 con 0x7f3d780b07d0 2023-10-06T14:26:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.301+0000 7f3d5f7fe640 1 --2- 172.21.15.19:0/1597526183 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3d6406c210 0x7f3d6406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.301+0000 7f3d7659c640 1 --2- 172.21.15.19:0/1597526183 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3d6406c210 0x7f3d6406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.301+0000 7f3d7659c640 10 monclient: get_auth_request con 0x7f3d6406c210 auth_method 0 2023-10-06T14:26:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.301+0000 7f3d7659c640 1 --2- 172.21.15.19:0/1597526183 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3d6406c210 0x7f3d6406e6c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3d780a93e0 tx=0x7f3d6c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.301+0000 7f3d5f7fe640 10 client.20800.objecter ms_handle_connect 0x7f3d6406c210 2023-10-06T14:26:33.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.303+0000 7f3d5f7fe640 1 -- 172.21.15.19:0/1597526183 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(970..970 src has 595..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f3d70057930 con 0x7f3d780b07d0 2023-10-06T14:26:33.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.303+0000 7f3d5f7fe640 10 client.20800.objecter ms_dispatch 0x7f3d78000b90 osd_map(970..970 src has 595..970) v4 2023-10-06T14:26:33.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.303+0000 7f3d5f7fe640 3 client.20800.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:33.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.303+0000 7f3d5f7fe640 3 client.20800.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:33.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.303+0000 7f3d5f7fe640 20 client.20800.objecter dump_active .. 0 homeless 2023-10-06T14:26:33.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.533+0000 7f3d5f7fe640 1 -- 172.21.15.19:0/1597526183 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f3d70053c70 con 0x7f3d780b07d0 2023-10-06T14:26:33.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.533+0000 7f3d5f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:33.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.533+0000 7f3d5f7fe640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:26:33.593 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:36.716 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:33.648 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:36.771 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:33.665 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:36.788 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:33.698 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:36.795 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:33.700 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:36.797 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:33.703 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:36.825 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:33.704 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:36.827 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:33.714 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:36.811 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:33.723 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:36.845 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.740+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1597526183 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d780af3f0 con 0x7f3d6406c210 2023-10-06T14:26:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.740+0000 7f3d5f7fe640 1 -- 172.21.15.19:0/1597526183 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3d6c0014a0 con 0x7f3d6406c210 2023-10-06T14:26:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T14:26:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 10 librados: watch_flush enter 2023-10-06T14:26:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 10 librados: watch_flush exit 2023-10-06T14:26:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 20 client.20800.objecter shutdown clearing up homeless session... 2023-10-06T14:26:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 10 client.20800.objecter successfully canceled tick 2023-10-06T14:26:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1597526183 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3d6406c210 msgr2=0x7f3d6406e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 1 --2- 172.21.15.19:0/1597526183 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3d6406c210 0x7f3d6406e6c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3d780a93e0 tx=0x7f3d6c009040 comp rx=0 tx=0).stop 2023-10-06T14:26:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 10 monclient: shutdown 2023-10-06T14:26:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1597526183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d780b07d0 msgr2=0x7f3d7813d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 1 --2- 172.21.15.19:0/1597526183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d780b07d0 0x7f3d7813d930 secure :-1 s=READY pgs=5123 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000c460 tx=0x7f3d7000c920 comp rx=0 tx=0).stop 2023-10-06T14:26:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1597526183 shutdown_connections 2023-10-06T14:26:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 1 --2- 172.21.15.19:0/1597526183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d780b3d60 0x7f3d781d9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 1 --2- 172.21.15.19:0/1597526183 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f3d6406c210 0x7f3d6406e6c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 1 --2- 172.21.15.19:0/1597526183 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3d780b3990 0x7f3d7813abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 1 --2- 172.21.15.19:0/1597526183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d780b07d0 0x7f3d7813d930 unknown :-1 s=CLOSED pgs=5123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1597526183 >> 172.21.15.19:0/1597526183 conn(0x7f3d780b6e00 msgr2=0x7f3d78140620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1597526183 shutdown_connections 2023-10-06T14:26:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 1 -- 172.21.15.19:0/1597526183 wait complete. 2023-10-06T14:26:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.742+0000 7f3d7d15c640 1 librados: shutdown 2023-10-06T14:26:33.759 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T14:26:33.759 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T14:26:33.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.882+0000 7f247f61f640 10 monclient: get_monmap_and_config 2023-10-06T14:26:33.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.882+0000 7f247f61f640 10 monclient: build_initial_monmap 2023-10-06T14:26:33.885 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.882+0000 7f247f61f640 10 monclient: monmap: 2023-10-06T14:26:33.886 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:33.886 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:33.886 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:33.883595+0000 2023-10-06T14:26:33.886 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:33.883595+0000 2023-10-06T14:26:33.886 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:33.886 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.886 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:33.886 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:33.887 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.883+0000 7f247f61f640 1 Processor -- start 2023-10-06T14:26:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.883+0000 7f247f61f640 1 -- start start 2023-10-06T14:26:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.883+0000 7f247f61f640 10 monclient: init 2023-10-06T14:26:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.883+0000 7f247f61f640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.883+0000 7f247f61f640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2023-10-06T14:26:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.883+0000 7f247f61f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f24780b3990 0x7f24780af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.884+0000 7f247f61f640 10 monclient(hunting): picked mon.noname-c con 0x7f24780b3990 addr v2:172.21.15.83:3300/0 2023-10-06T14:26:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.884+0000 7f247f61f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f24780b3d60 addr v1:172.21.15.19:6789/0 2023-10-06T14:26:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.884+0000 7f247f61f640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f24780b00c0 0x7f24780b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.884+0000 7f247f61f640 10 monclient(hunting): picked mon.noname-b con 0x7f24780b00c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.884+0000 7f247f61f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24780aebe0 con 0x7f24780b3d60 2023-10-06T14:26:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.884+0000 7f247f61f640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.884+0000 7f247f61f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24780afd50 con 0x7f24780b3990 2023-10-06T14:26:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.884+0000 7f247f61f640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.884+0000 7f247f61f640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2478140dd0 con 0x7f24780b00c0 2023-10-06T14:26:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.884+0000 7f247f61f640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:33.890 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.884+0000 7f247f61f640 10 monclient(hunting): authenticate will time out at 3322.070447s 2023-10-06T14:26:33.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.889+0000 7f247cbba640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f24780b3d60 0x7f2478141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48088/0 (socket says 172.21.15.19:48088) 2023-10-06T14:26:33.891 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.889+0000 7f247cbba640 1 -- 172.21.15.19:0/1251657802 learned_addr learned my addr 172.21.15.19:0/1251657802 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T14:26:33.900 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.898+0000 7f247d3bb640 1 --2- 172.21.15.19:0/1251657802 >> v2:172.21.15.83:3300/0 conn(0x7f24780b3990 0x7f24780af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.901+0000 7f247dbbc640 1 --2- 172.21.15.19:0/1251657802 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f24780b00c0 0x7f24780b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.904 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.902+0000 7f247d3bb640 10 monclient(hunting): get_auth_request con 0x7f24780b3990 auth_method 0 2023-10-06T14:26:33.904 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.902+0000 7f247d3bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:33.904 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.902+0000 7f247d3bb640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.904 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.902+0000 7f247d3bb640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.913+0000 7f246ffff640 1 -- 172.21.15.19:0/1251657802 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255921106 0 0) 0x7f24780aebe0 con 0x7f24780b3d60 2023-10-06T14:26:33.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.913+0000 7f246ffff640 10 monclient(hunting): _negotiate 2023-10-06T14:26:33.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.913+0000 7f246ffff640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.913+0000 7f246ffff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.915 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.913+0000 7f246ffff640 10 monclient(hunting): my global_id is 15687 2023-10-06T14:26:33.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.913+0000 7f246ffff640 1 -- 172.21.15.19:0/1251657802 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2464003520 con 0x7f24780b3d60 2023-10-06T14:26:33.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.915+0000 7f247dbbc640 10 monclient(hunting): get_auth_request con 0x7f24780b00c0 auth_method 0 2023-10-06T14:26:33.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.915+0000 7f247dbbc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:33.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.915+0000 7f247dbbc640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.918 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.915+0000 7f247dbbc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.920+0000 7f247d3bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.920+0000 7f247d3bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.920+0000 7f247d3bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:33.927 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 -- 172.21.15.19:0/1251657802 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 142050469 0 0) 0x7f2464003520 con 0x7f24780b3d60 2023-10-06T14:26:33.927 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 -- 172.21.15.19:0/1251657802 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f24780b00c0 msgr2=0x7f24780b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:33.927 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 --2- 172.21.15.19:0/1251657802 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f24780b00c0 0x7f24780b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.927 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 -- 172.21.15.19:0/1251657802 >> v2:172.21.15.83:3300/0 conn(0x7f24780b3990 msgr2=0x7f24780af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:33.927 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 --2- 172.21.15.19:0/1251657802 >> v2:172.21.15.83:3300/0 conn(0x7f24780b3990 0x7f24780af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.927 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient: _finish_hunting 0 2023-10-06T14:26:33.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 monclient: found mon.noname-a-legacy 2023-10-06T14:26:33.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:33.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.19:6789/0 2023-10-06T14:26:33.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 -- 172.21.15.19:0/1251657802 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2478141070 con 0x7f24780b3d60 2023-10-06T14:26:33.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient: _finish_auth 0 2023-10-06T14:26:33.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient: _check_auth_tickets 2023-10-06T14:26:33.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:33.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 -- 172.21.15.19:0/1251657802 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251531364 0 0) 0x7f2468001670 con 0x7f24780b3d60 2023-10-06T14:26:33.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f247f61f640 5 monclient: authenticate success, global_id 15687 2023-10-06T14:26:33.928 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f247f61f640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:33.929 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:33.929 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2023-10-06T14:26:33.929 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient: dump: 2023-10-06T14:26:33.929 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:33.929 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:33.929 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.929 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.929 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:33.929 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.929 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:33.929 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:33.929 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:33.930 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] but i'm connected to v1:172.21.15.19:6789/0, reconnecting 2023-10-06T14:26:33.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:33.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 -- 172.21.15.19:0/1251657802 >> v1:172.21.15.19:6789/0 conn(0x7f24780b3d60 legacy=0x7f2478141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T14:26:33.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 --2- 172.21.15.19:0/1251657802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464003970 0x7f2464003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient(hunting): picked mon.a con 0x7f2464003970 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:33.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 --2- 172.21.15.19:0/1251657802 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f24780b00c0 0x7f24640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient(hunting): picked mon.c con 0x7f24780b00c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:33.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 --2- 172.21.15.19:0/1251657802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2464004d00 0x7f2464005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.930 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient(hunting): picked mon.b con 0x7f2464004d00 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:33.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 -- 172.21.15.19:0/1251657802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24780afd50 con 0x7f2464003970 2023-10-06T14:26:33.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 -- 172.21.15.19:0/1251657802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2478140dd0 con 0x7f2464004d00 2023-10-06T14:26:33.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 1 -- 172.21.15.19:0/1251657802 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246400ba00 con 0x7f24780b00c0 2023-10-06T14:26:33.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:33.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f246ffff640 10 monclient(hunting): _finish_auth 0 2023-10-06T14:26:33.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.924+0000 7f247f61f640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.932+0000 7f247d3bb640 1 --2- 172.21.15.19:0/1251657802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464003970 0x7f2464003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.942+0000 7f247d3bb640 10 monclient(hunting): get_auth_request con 0x7f2464003970 auth_method 0 2023-10-06T14:26:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.942+0000 7f247d3bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.942+0000 7f247d3bb640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.942+0000 7f247d3bb640 10 monclient(hunting): _init_auth already have auth, reseting 2023-10-06T14:26:33.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.944+0000 7f247d3bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:33.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.944+0000 7f247d3bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:33.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.944+0000 7f247d3bb640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-10-06T14:26:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.952+0000 7f247d3bb640 10 monclient(hunting): handle_auth_done global_id 15687 payload 293 2023-10-06T14:26:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.952+0000 7f247d3bb640 1 -- 172.21.15.19:0/1251657802 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f24780b00c0 msgr2=0x7f24640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.952+0000 7f247d3bb640 1 --2- 172.21.15.19:0/1251657802 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f24780b00c0 0x7f24640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.952+0000 7f247d3bb640 1 -- 172.21.15.19:0/1251657802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2464004d00 msgr2=0x7f2464005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.952+0000 7f247d3bb640 1 --2- 172.21.15.19:0/1251657802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2464004d00 0x7f2464005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.952+0000 7f247d3bb640 10 monclient: _finish_hunting 0 2023-10-06T14:26:33.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.952+0000 7f247d3bb640 1 monclient: found mon.a 2023-10-06T14:26:33.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.952+0000 7f247d3bb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:33.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.952+0000 7f247d3bb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T14:26:33.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.952+0000 7f247d3bb640 1 -- 172.21.15.19:0/1251657802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2464003520 con 0x7f2464003970 2023-10-06T14:26:33.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.952+0000 7f247d3bb640 10 monclient: _finish_auth 0 2023-10-06T14:26:33.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.952+0000 7f247d3bb640 10 monclient: _check_auth_tickets 2023-10-06T14:26:33.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.952+0000 7f247d3bb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:33.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.962+0000 7f247d3bb640 1 --2- 172.21.15.19:0/1251657802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464003970 0x7f2464003d80 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 crypto rx=0x7f2474009ac0 tx=0x7f24640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e8f0f788a6b35b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:33.977 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.975+0000 7f246ffff640 1 -- 172.21.15.19:0/1251657802 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247402c710 con 0x7f2464003970 2023-10-06T14:26:33.978 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.975+0000 7f246ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:33.978 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.975+0000 7f246ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T14:26:33.978 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.975+0000 7f246ffff640 10 monclient: dump: 2023-10-06T14:26:33.978 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:33.978 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:33.978 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.978 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.978 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:33.978 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.978 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:33.979 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:33.979 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:33.979 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.975+0000 7f246ffff640 1 -- 172.21.15.19:0/1251657802 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247402c870 con 0x7f2464003970 2023-10-06T14:26:33.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.975+0000 7f246ffff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:33.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.975+0000 7f246ffff640 1 -- 172.21.15.19:0/1251657802 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247402cc00 con 0x7f2464003970 2023-10-06T14:26:33.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.975+0000 7f247f61f640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:33.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.975+0000 7f247f61f640 4 set_mon_vals no callback set 2023-10-06T14:26:33.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.975+0000 7f246ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:33.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.975+0000 7f246ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T14:26:33.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.975+0000 7f246ffff640 10 monclient: dump: 2023-10-06T14:26:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:33.980 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.976+0000 7f247f61f640 10 monclient: shutdown 2023-10-06T14:26:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.976+0000 7f247f61f640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.976+0000 7f247f61f640 1 -- 172.21.15.19:0/1251657802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464003970 msgr2=0x7f2464003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.976+0000 7f247f61f640 1 --2- 172.21.15.19:0/1251657802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464003970 0x7f2464003d80 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 crypto rx=0x7f2474009ac0 tx=0x7f24640057f0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.976+0000 7f247dbbc640 1 -- 172.21.15.19:0/1251657802 reap_dead start 2023-10-06T14:26:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.976+0000 7f247f61f640 1 -- 172.21.15.19:0/1251657802 shutdown_connections 2023-10-06T14:26:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.976+0000 7f247f61f640 1 -- 172.21.15.19:0/1251657802 >> 172.21.15.19:0/1251657802 conn(0x7f24780b6e00 msgr2=0x7f24780b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.976+0000 7f247f61f640 1 -- 172.21.15.19:0/1251657802 shutdown_connections 2023-10-06T14:26:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.976+0000 7f247f61f640 1 -- 172.21.15.19:0/1251657802 wait complete. 2023-10-06T14:26:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.977+0000 7f247f61f640 10 monclient: build_initial_monmap 2023-10-06T14:26:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.977+0000 7f247f61f640 10 monclient: monmap: 2023-10-06T14:26:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:33.984 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.977+0000 7f247f61f640 1 librados: starting msgr at 2023-10-06T14:26:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.977+0000 7f247f61f640 1 librados: starting objecter 2023-10-06T14:26:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.977+0000 7f247f61f640 1 Processor -- start 2023-10-06T14:26:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.977+0000 7f247f61f640 1 -- start start 2023-10-06T14:26:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.977+0000 7f247f61f640 1 librados: setting wanted keys 2023-10-06T14:26:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.977+0000 7f247f61f640 1 librados: calling monclient init 2023-10-06T14:26:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.977+0000 7f247f61f640 10 monclient: init 2023-10-06T14:26:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 10 monclient: _add_conns ranks=[2,1,0] 2023-10-06T14:26:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2464003970 0x7f24781d9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 10 monclient(hunting): picked mon.noname-c con 0x7f2464003970 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24781d9d10 0x7f24780abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 10 monclient(hunting): picked mon.noname-b con 0x7f24781d9d10 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24780ac030 0x7f24781d5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 10 monclient(hunting): picked mon.noname-a con 0x7f24780ac030 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24780ab6d0 con 0x7f24780ac030 2023-10-06T14:26:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24781d3730 con 0x7f24781d9d10 2023-10-06T14:26:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24780aaa90 con 0x7f2464003970 2023-10-06T14:26:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.978+0000 7f247f61f640 10 monclient(hunting): authenticate will time out at 3322.164790s 2023-10-06T14:26:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.981+0000 7f247d3bb640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2464003970 0x7f24781d9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.983+0000 7f247dbbc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24780ac030 0x7f24781d5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.984+0000 7f247d3bb640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2464003970 0x7f24781d9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40468/0 (socket says 172.21.15.19:40468) 2023-10-06T14:26:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.984+0000 7f247d3bb640 1 -- 172.21.15.19:0/2677754197 learned_addr learned my addr 172.21.15.19:0/2677754197 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.984+0000 7f247d3bb640 10 monclient(hunting): get_auth_request con 0x7f2464003970 auth_method 0 2023-10-06T14:26:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.984+0000 7f247d3bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.984+0000 7f247d3bb640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.984+0000 7f247d3bb640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.986+0000 7f247d3bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.986+0000 7f247d3bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:33.989 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.986+0000 7f247d3bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.988+0000 7f247dbbc640 10 monclient(hunting): get_auth_request con 0x7f24780ac030 auth_method 0 2023-10-06T14:26:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.988+0000 7f247dbbc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.988+0000 7f247dbbc640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.988+0000 7f247dbbc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.989+0000 7f247dbbc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.989+0000 7f247dbbc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.989+0000 7f247dbbc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.992+0000 7f247dbbc640 10 monclient(hunting): handle_auth_done global_id 15690 payload 824 2023-10-06T14:26:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.992+0000 7f247dbbc640 1 -- 172.21.15.19:0/2677754197 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2464003970 msgr2=0x7f24781d9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.992+0000 7f247dbbc640 1 --2- 172.21.15.19:0/2677754197 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f2464003970 0x7f24781d9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.992+0000 7f247dbbc640 1 -- 172.21.15.19:0/2677754197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24781d9d10 msgr2=0x7f24780abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.992+0000 7f247dbbc640 1 --2- 172.21.15.19:0/2677754197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24781d9d10 0x7f24780abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.992+0000 7f247dbbc640 10 monclient: _finish_hunting 0 2023-10-06T14:26:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.992+0000 7f247dbbc640 1 monclient: found mon.noname-a 2023-10-06T14:26:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.992+0000 7f247dbbc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.992+0000 7f247dbbc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T14:26:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:33.992+0000 7f247dbbc640 1 -- 172.21.15.19:0/2677754197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247813c970 con 0x7f24780ac030 2023-10-06T14:26:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.003+0000 7f247dbbc640 1 --2- 172.21.15.19:0/2677754197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24780ac030 0x7f24781d5b10 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 crypto rx=0x7f24780722b0 tx=0x7f2470007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.003+0000 7f246dffb640 10 client.?.objecter ms_handle_connect 0x7f24780ac030 2023-10-06T14:26:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.003+0000 7f246dffb640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:34.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f246dffb640 1 -- 172.21.15.19:0/2677754197 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470012710 con 0x7f24780ac030 2023-10-06T14:26:34.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f246dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:34.025 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f246dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T14:26:34.026 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f246dffb640 10 monclient: dump: 2023-10-06T14:26:34.026 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:34.026 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:34.026 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.026 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.026 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:34.026 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:34.026 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:34.026 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:34.026 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:34.026 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:34.027 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f246dffb640 10 monclient: _finish_auth 0 2023-10-06T14:26:34.027 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f246dffb640 10 monclient: _check_auth_tickets 2023-10-06T14:26:34.027 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f246dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:34.027 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f247f61f640 5 monclient: authenticate success, global_id 15690 2023-10-06T14:26:34.027 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f246dffb640 1 -- 172.21.15.19:0/2677754197 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2470012870 con 0x7f24780ac030 2023-10-06T14:26:34.027 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f247f61f640 10 monclient: _renew_subs 2023-10-06T14:26:34.027 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f247f61f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T14:26:34.027 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f247f61f640 1 -- 172.21.15.19:0/2677754197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24780ac4d0 con 0x7f24780ac030 2023-10-06T14:26:34.027 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f246dffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:34.027 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f246dffb640 1 -- 172.21.15.19:0/2677754197 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470012c20 con 0x7f24780ac030 2023-10-06T14:26:34.027 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f246dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:34.027 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f246dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T14:26:34.028 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f246dffb640 10 monclient: dump: 2023-10-06T14:26:34.028 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:34.028 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:34.028 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.028 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.028 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:34.028 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:34.028 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:34.028 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:34.028 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:34.028 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:34.028 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f246f7fe640 4 set_mon_vals no callback set 2023-10-06T14:26:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f247f61f640 10 client.15690.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f247f61f640 10 monclient: _renew_subs 2023-10-06T14:26:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f247f61f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T14:26:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f247f61f640 1 -- 172.21.15.19:0/2677754197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24781d6050 con 0x7f24780ac030 2023-10-06T14:26:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.022+0000 7f247f61f640 1 librados: init done 2023-10-06T14:26:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.023+0000 7f247f61f640 10 client.15690.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.023+0000 7f247f61f640 15 client.15690.objecter _session_command_op_assign -1 1 2023-10-06T14:26:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.023+0000 7f247f61f640 20 client.15690.objecter _get_session osd=-1 returning homeless 2023-10-06T14:26:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.023+0000 7f247f61f640 10 client.15690.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:34.031 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.023+0000 7f247f61f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T14:26:34.031 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.023+0000 7f247f61f640 1 -- 172.21.15.19:0/2677754197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f24780aac20 con 0x7f24780ac030 2023-10-06T14:26:34.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.064+0000 7f246dffb640 1 -- 172.21.15.19:0/2677754197 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f247001d3b0 con 0x7f24780ac030 2023-10-06T14:26:34.067 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.065+0000 7f246dffb640 1 --2- 172.21.15.19:0/2677754197 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f245c06c240 0x7f245c06e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.065+0000 7f247d3bb640 1 --2- 172.21.15.19:0/2677754197 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f245c06c240 0x7f245c06e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.065+0000 7f247d3bb640 10 monclient: get_auth_request con 0x7f245c06c240 auth_method 0 2023-10-06T14:26:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.065+0000 7f247d3bb640 1 --2- 172.21.15.19:0/2677754197 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f245c06c240 0x7f245c06e6f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2474000c00 tx=0x7f2474030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:34.069 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.065+0000 7f246dffb640 10 client.15690.objecter ms_handle_connect 0x7f245c06c240 2023-10-06T14:26:34.072 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.069+0000 7f246dffb640 1 -- 172.21.15.19:0/2677754197 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(970..970 src has 714..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f2470057b00 con 0x7f24780ac030 2023-10-06T14:26:34.072 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.069+0000 7f246dffb640 10 client.15690.objecter ms_dispatch 0x7f2478000b90 osd_map(970..970 src has 714..970) v4 2023-10-06T14:26:34.072 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.069+0000 7f246dffb640 3 client.15690.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.069+0000 7f246dffb640 3 client.15690.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.069+0000 7f246dffb640 10 client.15690.objecter checking command 1 2023-10-06T14:26:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.069+0000 7f246dffb640 1 --2- 172.21.15.19:0/2677754197 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x7f245c073a60 0x7f245c075ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.069+0000 7f246dffb640 20 client.15690.objecter _get_session s=0x7f245c073390 osd=6 3 2023-10-06T14:26:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.069+0000 7f246dffb640 20 client.15690.objecter put_session s=0x7f245c073390 osd=6 3 2023-10-06T14:26:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.070+0000 7f246dffb640 15 client.15690.objecter _session_command_op_remove -1 1 2023-10-06T14:26:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.070+0000 7f246dffb640 20 client.15690.objecter _get_session s=0x7f245c073390 osd=6 3 2023-10-06T14:26:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.070+0000 7f246dffb640 20 client.15690.objecter get_session s=0x7f245c073390 osd=6 3 2023-10-06T14:26:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.070+0000 7f246dffb640 15 client.15690.objecter _session_command_op_assign 6 1 2023-10-06T14:26:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.070+0000 7f246dffb640 20 client.15690.objecter put_session s=0x7f245c073390 osd=6 4 2023-10-06T14:26:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.070+0000 7f246dffb640 10 client.15690.objecter _send_command 1 2023-10-06T14:26:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.070+0000 7f246dffb640 1 -- 172.21.15.19:0/2677754197 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f245c0765c0 con 0x7f245c073a60 2023-10-06T14:26:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.070+0000 7f246dffb640 20 client.15690.objecter dump_active .. 0 homeless 2023-10-06T14:26:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.070+0000 7f246dffb640 1 -- 172.21.15.19:0/2677754197 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=970) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2470053e40 con 0x7f24780ac030 2023-10-06T14:26:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.070+0000 7f246dffb640 10 monclient: handle_get_version_reply finishing 1 version 970 2023-10-06T14:26:34.075 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.070+0000 7f247cbba640 1 --2- 172.21.15.19:0/2677754197 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x7f245c073a60 0x7f245c075ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:34.075 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.071+0000 7f247cbba640 10 monclient: get_auth_request con 0x7f245c073a60 auth_method 0 2023-10-06T14:26:34.075 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.073+0000 7f247cbba640 1 --2- 172.21.15.19:0/2677754197 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x7f245c073a60 0x7f245c075ef0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.073+0000 7f246dffb640 10 client.15690.objecter ms_handle_connect 0x7f245c073a60 2023-10-06T14:26:34.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.097+0000 7f246dffb640 1 -- 172.21.15.19:0/2677754197 <== osd.6 v2:172.21.15.83:6802/86124 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f245c0765c0 con 0x7f245c073a60 2023-10-06T14:26:34.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.097+0000 7f246dffb640 10 client.15690.objecter ms_dispatch 0x7f2478000b90 command_reply(tid 1: 0 ) v1 2023-10-06T14:26:34.099 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.097+0000 7f246dffb640 10 client.15690.objecter _finish_command 1 = system:0 2023-10-06T14:26:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.097+0000 7f246dffb640 20 client.15690.objecter put_session s=0x7f245c073390 osd=6 4 2023-10-06T14:26:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.097+0000 7f246dffb640 15 client.15690.objecter _session_command_op_remove 6 1 2023-10-06T14:26:34.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.113+0000 7f247f61f640 10 client.15690.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-10-06T14:26:34.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.113+0000 7f247f61f640 15 client.15690.objecter _session_command_op_assign -1 2 2023-10-06T14:26:34.115 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.113+0000 7f247f61f640 20 client.15690.objecter _get_session s=0x7f245c073390 osd=6 3 2023-10-06T14:26:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.113+0000 7f247f61f640 20 client.15690.objecter put_session s=0x7f245c073390 osd=6 3 2023-10-06T14:26:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.113+0000 7f247f61f640 20 client.15690.objecter _get_session s=0x7f245c073390 osd=6 3 2023-10-06T14:26:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.113+0000 7f247f61f640 15 client.15690.objecter _session_command_op_remove -1 2 2023-10-06T14:26:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.113+0000 7f247f61f640 20 client.15690.objecter get_session s=0x7f245c073390 osd=6 3 2023-10-06T14:26:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.113+0000 7f247f61f640 15 client.15690.objecter _session_command_op_assign 6 2 2023-10-06T14:26:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.113+0000 7f247f61f640 20 client.15690.objecter put_session s=0x7f245c073390 osd=6 4 2023-10-06T14:26:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.113+0000 7f247f61f640 10 client.15690.objecter _send_command 2 2023-10-06T14:26:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.113+0000 7f247f61f640 1 -- 172.21.15.19:0/2677754197 --> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f24780af110 con 0x7f245c073a60 2023-10-06T14:26:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.127+0000 7f246dffb640 1 -- 172.21.15.19:0/2677754197 <== osd.6 v2:172.21.15.83:6802/86124 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2468007b60 con 0x7f245c073a60 2023-10-06T14:26:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.127+0000 7f246dffb640 10 client.15690.objecter ms_dispatch 0x7f2478000b90 command_reply(tid 2: 0 ) v1 2023-10-06T14:26:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.127+0000 7f246dffb640 10 client.15690.objecter _finish_command 2 = system:0 2023-10-06T14:26:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.127+0000 7f246dffb640 20 client.15690.objecter put_session s=0x7f245c073390 osd=6 4 2023-10-06T14:26:34.130 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.127+0000 7f246dffb640 15 client.15690.objecter _session_command_op_remove 6 2 2023-10-06T14:26:34.131 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2023-10-06T14:26:34.131 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T14:26:34.131 INFO:teuthology.orchestra.run.smithi019.stdout:} 2023-10-06T14:26:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 10 librados: watch_flush enter 2023-10-06T14:26:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 10 librados: watch_flush exit 2023-10-06T14:26:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 10 client.15690.objecter close_session for osd.6 2023-10-06T14:26:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 1 -- 172.21.15.19:0/2677754197 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x7f245c073a60 msgr2=0x7f245c075ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 1 --2- 172.21.15.19:0/2677754197 >> [v2:172.21.15.83:6802/86124,v1:172.21.15.83:6803/86124] conn(0x7f245c073a60 0x7f245c075ef0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 20 client.15690.objecter put_session s=0x7f245c073390 osd=6 1 2023-10-06T14:26:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 20 client.15690.objecter shutdown clearing up homeless session... 2023-10-06T14:26:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 10 client.15690.objecter successfully canceled tick 2023-10-06T14:26:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 1 -- 172.21.15.19:0/2677754197 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f245c06c240 msgr2=0x7f245c06e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 1 --2- 172.21.15.19:0/2677754197 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f245c06c240 0x7f245c06e6f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2474000c00 tx=0x7f2474030000 comp rx=0 tx=0).stop 2023-10-06T14:26:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 10 monclient: shutdown 2023-10-06T14:26:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 1 -- 172.21.15.19:0/2677754197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24780ac030 msgr2=0x7f24781d5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 1 --2- 172.21.15.19:0/2677754197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24780ac030 0x7f24781d5b10 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 crypto rx=0x7f24780722b0 tx=0x7f2470007d80 comp rx=0 tx=0).stop 2023-10-06T14:26:34.133 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247dbbc640 1 -- 172.21.15.19:0/2677754197 reap_dead start 2023-10-06T14:26:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 1 -- 172.21.15.19:0/2677754197 shutdown_connections 2023-10-06T14:26:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 1 -- 172.21.15.19:0/2677754197 >> 172.21.15.19:0/2677754197 conn(0x7f24780b6e00 msgr2=0x7f247813e490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 1 -- 172.21.15.19:0/2677754197 shutdown_connections 2023-10-06T14:26:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 1 -- 172.21.15.19:0/2677754197 wait complete. 2023-10-06T14:26:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.128+0000 7f247f61f640 1 librados: shutdown 2023-10-06T14:26:34.143 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T14:26:34.258 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.256+0000 7f555d531640 10 monclient: get_monmap_and_config 2023-10-06T14:26:34.258 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.256+0000 7f555d531640 10 monclient: build_initial_monmap 2023-10-06T14:26:34.258 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.256+0000 7f555d531640 10 monclient: monmap: 2023-10-06T14:26:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:34.257100+0000 2023-10-06T14:26:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:34.257100+0000 2023-10-06T14:26:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:34.260 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.256+0000 7f555d531640 1 Processor -- start 2023-10-06T14:26:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.256+0000 7f555d531640 1 -- start start 2023-10-06T14:26:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.256+0000 7f555d531640 10 monclient: init 2023-10-06T14:26:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.257+0000 7f555d531640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.257+0000 7f555d531640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2023-10-06T14:26:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.257+0000 7f555d531640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f55580b3990 0x7f55580af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.257+0000 7f555d531640 10 monclient(hunting): picked mon.noname-a con 0x7f55580b3990 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.257+0000 7f555d531640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f55580b3d60 addr v1:172.21.15.83:6789/0 2023-10-06T14:26:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.257+0000 7f555d531640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f55580b00c0 addr v1:172.21.15.19:6789/0 2023-10-06T14:26:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.257+0000 7f555d531640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55580aebe0 con 0x7f55580b00c0 2023-10-06T14:26:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.257+0000 7f555d531640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5558141070 con 0x7f55580b3d60 2023-10-06T14:26:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.257+0000 7f555d531640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.257+0000 7f555d531640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5558140dd0 con 0x7f55580b3990 2023-10-06T14:26:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.257+0000 7f555d531640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.257+0000 7f555d531640 10 monclient(hunting): authenticate will time out at 3322.443831s 2023-10-06T14:26:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.267+0000 7f5556ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f55580b3990 0x7f55580af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.267+0000 7f55567fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f55580b3d60 0x7f5558141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:43016/0 (socket says 172.21.15.19:43016) 2023-10-06T14:26:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.267+0000 7f55567fc640 1 -- 172.21.15.19:0/2203802251 learned_addr learned my addr 172.21.15.19:0/2203802251 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T14:26:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.270+0000 7f5556ffd640 10 monclient(hunting): get_auth_request con 0x7f55580b3990 auth_method 0 2023-10-06T14:26:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.270+0000 7f5556ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.270+0000 7f5556ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.270+0000 7f5556ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.271+0000 7f5555ffb640 1 -- 172.21.15.19:0/2203802251 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670181061 0 0) 0x7f5558141070 con 0x7f55580b3d60 2023-10-06T14:26:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.271+0000 7f5555ffb640 10 monclient(hunting): _negotiate 2023-10-06T14:26:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.271+0000 7f5555ffb640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.271+0000 7f5555ffb640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.271+0000 7f5555ffb640 10 monclient(hunting): my global_id is 20806 2023-10-06T14:26:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.271+0000 7f5555ffb640 1 -- 172.21.15.19:0/2203802251 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5540003520 con 0x7f55580b3d60 2023-10-06T14:26:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.273+0000 7f5556ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.273+0000 7f5556ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:34.275 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.273+0000 7f5556ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:34.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.277+0000 7f5556ffd640 10 monclient(hunting): handle_auth_done global_id 15693 payload 274 2023-10-06T14:26:34.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.277+0000 7f5556ffd640 1 -- 172.21.15.19:0/2203802251 >> v1:172.21.15.83:6789/0 conn(0x7f55580b3d60 legacy=0x7f5558141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:34.279 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.277+0000 7f5556ffd640 1 -- 172.21.15.19:0/2203802251 >> v1:172.21.15.19:6789/0 conn(0x7f55580b00c0 legacy=0x7f555814b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:34.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.277+0000 7f5556ffd640 10 monclient: _finish_hunting 0 2023-10-06T14:26:34.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.277+0000 7f5556ffd640 1 monclient: found mon.noname-a 2023-10-06T14:26:34.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.277+0000 7f5556ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:34.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.277+0000 7f5556ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T14:26:34.280 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.277+0000 7f5556ffd640 1 -- 172.21.15.19:0/2203802251 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5558141c10 con 0x7f55580b3990 2023-10-06T14:26:34.290 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.288+0000 7f5556ffd640 1 --2- 172.21.15.19:0/2203802251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55580b3990 0x7f55580af810 secure :-1 s=READY pgs=3196 cs=0 l=1 rev1=1 crypto rx=0x7f55480097a0 tx=0x7f554802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa44fd0a85a241e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f5555ffb640 1 -- 172.21.15.19:0/2203802251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554802e070 con 0x7f55580b3990 2023-10-06T14:26:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f5555ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f5555ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T14:26:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f5555ffb640 10 monclient: dump: 2023-10-06T14:26:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:34.302 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f5555ffb640 10 monclient: _finish_auth 0 2023-10-06T14:26:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f5555ffb640 10 monclient: _check_auth_tickets 2023-10-06T14:26:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f5555ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f5555ffb640 1 -- 172.21.15.19:0/2203802251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55480315a0 con 0x7f55580b3990 2023-10-06T14:26:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f555d531640 5 monclient: authenticate success, global_id 15693 2023-10-06T14:26:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f555d531640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f5555ffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f5555ffb640 1 -- 172.21.15.19:0/2203802251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548031930 con 0x7f55580b3990 2023-10-06T14:26:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f5555ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f5555ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T14:26:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f5555ffb640 10 monclient: dump: 2023-10-06T14:26:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:34.304 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f555d531640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.299+0000 7f555d531640 4 set_mon_vals no callback set 2023-10-06T14:26:34.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.300+0000 7f555d531640 10 monclient: shutdown 2023-10-06T14:26:34.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.300+0000 7f555d531640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:34.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.300+0000 7f555d531640 1 -- 172.21.15.19:0/2203802251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55580b3990 msgr2=0x7f55580af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:34.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.300+0000 7f555d531640 1 --2- 172.21.15.19:0/2203802251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55580b3990 0x7f55580af810 secure :-1 s=READY pgs=3196 cs=0 l=1 rev1=1 crypto rx=0x7f55480097a0 tx=0x7f554802b840 comp rx=0 tx=0).stop 2023-10-06T14:26:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.300+0000 7f555d531640 1 -- 172.21.15.19:0/2203802251 shutdown_connections 2023-10-06T14:26:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.300+0000 7f555d531640 1 --2- 172.21.15.19:0/2203802251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55580b3990 0x7f55580af810 unknown :-1 s=CLOSED pgs=3196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.300+0000 7f555d531640 1 -- 172.21.15.19:0/2203802251 >> 172.21.15.19:0/2203802251 conn(0x7f55580b6e00 msgr2=0x7f55580b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.300+0000 7f555d531640 1 -- 172.21.15.19:0/2203802251 shutdown_connections 2023-10-06T14:26:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.300+0000 7f555d531640 1 -- 172.21.15.19:0/2203802251 wait complete. 2023-10-06T14:26:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.300+0000 7f555d531640 10 monclient: build_initial_monmap 2023-10-06T14:26:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.300+0000 7f555d531640 10 monclient: monmap: 2023-10-06T14:26:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:34.308 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:34.308 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:34.308 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:34.308 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:34.308 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:34.308 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.301+0000 7f555d531640 1 librados: starting msgr at 2023-10-06T14:26:34.308 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.301+0000 7f555d531640 1 librados: starting objecter 2023-10-06T14:26:34.308 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.301+0000 7f555d531640 1 Processor -- start 2023-10-06T14:26:34.308 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.301+0000 7f555d531640 1 -- start start 2023-10-06T14:26:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.301+0000 7f555d531640 1 librados: setting wanted keys 2023-10-06T14:26:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.301+0000 7f555d531640 1 librados: calling monclient init 2023-10-06T14:26:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.301+0000 7f555d531640 10 monclient: init 2023-10-06T14:26:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 10 monclient: _add_conns ranks=[2,1,0] 2023-10-06T14:26:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f55580b00c0 0x7f5558144610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 10 monclient(hunting): picked mon.noname-c con 0x7f55580b00c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55580b3990 0x7f55580ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 10 monclient(hunting): picked mon.noname-b con 0x7f55580b3990 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55580b3d60 0x7f55581d67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 10 monclient(hunting): picked mon.noname-a con 0x7f55580b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55581d6080 con 0x7f55580b3d60 2023-10-06T14:26:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55580ad550 con 0x7f55580b3990 2023-10-06T14:26:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555813ad90 con 0x7f55580b00c0 2023-10-06T14:26:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.302+0000 7f555d531640 10 monclient(hunting): authenticate will time out at 3322.488641s 2023-10-06T14:26:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.303+0000 7f5556ffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f55580b00c0 0x7f5558144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.304+0000 7f5556ffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f55580b00c0 0x7f5558144610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:40476/0 (socket says 172.21.15.19:40476) 2023-10-06T14:26:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.304+0000 7f5556ffd640 1 -- 172.21.15.19:0/3027487759 learned_addr learned my addr 172.21.15.19:0/3027487759 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.304+0000 7f5556ffd640 10 monclient(hunting): get_auth_request con 0x7f55580b00c0 auth_method 0 2023-10-06T14:26:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.304+0000 7f5556ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.304+0000 7f5556ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.304+0000 7f5556ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.305+0000 7f5556ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.305+0000 7f5556ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.305+0000 7f5556ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.305+0000 7f55577fe640 1 --2- 172.21.15.19:0/3027487759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55580b3d60 0x7f55581d67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.306+0000 7f55577fe640 10 monclient(hunting): get_auth_request con 0x7f55580b3d60 auth_method 0 2023-10-06T14:26:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.306+0000 7f55577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.306+0000 7f55577fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.306+0000 7f55577fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.309+0000 7f5556ffd640 10 monclient(hunting): handle_auth_done global_id 17198 payload 824 2023-10-06T14:26:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.309+0000 7f5556ffd640 1 -- 172.21.15.19:0/3027487759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55580b3990 msgr2=0x7f55580ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.309+0000 7f5556ffd640 1 --2- 172.21.15.19:0/3027487759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55580b3990 0x7f55580ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.309+0000 7f5556ffd640 1 -- 172.21.15.19:0/3027487759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55580b3d60 msgr2=0x7f55581d67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.309+0000 7f5556ffd640 1 --2- 172.21.15.19:0/3027487759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55580b3d60 0x7f55581d67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.309+0000 7f5556ffd640 10 monclient: _finish_hunting 0 2023-10-06T14:26:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.309+0000 7f5556ffd640 1 monclient: found mon.noname-c 2023-10-06T14:26:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.309+0000 7f5556ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.309+0000 7f5556ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3301/0 2023-10-06T14:26:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.309+0000 7f5556ffd640 1 -- 172.21.15.19:0/3027487759 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5548009400 con 0x7f55580b00c0 2023-10-06T14:26:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.313+0000 7f5556ffd640 1 --2- 172.21.15.19:0/3027487759 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f55580b00c0 0x7f5558144610 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 crypto rx=0x7f5548000c00 tx=0x7f55480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.313+0000 7f553f7fe640 10 client.?.objecter ms_handle_connect 0x7f55580b00c0 2023-10-06T14:26:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.313+0000 7f553f7fe640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:34.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f553f7fe640 1 -- 172.21.15.19:0/3027487759 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548032840 con 0x7f55580b00c0 2023-10-06T14:26:34.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f553f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:34.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f553f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T14:26:34.322 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f553f7fe640 10 monclient: dump: 2023-10-06T14:26:34.322 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:34.322 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:34.322 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.322 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.322 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:34.323 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:34.323 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:34.323 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:34.323 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:34.323 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:34.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f553f7fe640 10 monclient: _finish_auth 0 2023-10-06T14:26:34.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f553f7fe640 10 monclient: _check_auth_tickets 2023-10-06T14:26:34.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f553f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:34.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f553f7fe640 1 -- 172.21.15.19:0/3027487759 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55480329a0 con 0x7f55580b00c0 2023-10-06T14:26:34.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f555d531640 5 monclient: authenticate success, global_id 17198 2023-10-06T14:26:34.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f555d531640 10 monclient: _renew_subs 2023-10-06T14:26:34.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f555d531640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:34.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f555d531640 1 -- 172.21.15.19:0/3027487759 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55581d6330 con 0x7f55580b00c0 2023-10-06T14:26:34.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f553f7fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:34.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f555d531640 10 client.17198.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:34.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f555d531640 10 monclient: _renew_subs 2023-10-06T14:26:34.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f553f7fe640 1 -- 172.21.15.19:0/3027487759 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548032d50 con 0x7f55580b00c0 2023-10-06T14:26:34.324 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f555d531640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:34.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f555d531640 1 -- 172.21.15.19:0/3027487759 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55581d6f10 con 0x7f55580b00c0 2023-10-06T14:26:34.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f555d531640 1 librados: init done 2023-10-06T14:26:34.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f553f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:34.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f553f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:26:34.325 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f553f7fe640 10 monclient: dump: 2023-10-06T14:26:34.325 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:34.325 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:34.325 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.325 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.325 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:34.325 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:34.325 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:34.326 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:34.326 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:34.326 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:34.326 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.319+0000 7f55557fa640 4 set_mon_vals no callback set 2023-10-06T14:26:34.327 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.320+0000 7f555d531640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:34.327 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.320+0000 7f555d531640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:34.327 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.320+0000 7f555d531640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3301/0 2023-10-06T14:26:34.327 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.320+0000 7f555d531640 1 -- 172.21.15.19:0/3027487759 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55580678e0 con 0x7f55580b00c0 2023-10-06T14:26:34.629 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.626+0000 7f553f7fe640 1 -- 172.21.15.19:0/3027487759 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f554804d3f0 con 0x7f55580b00c0 2023-10-06T14:26:34.629 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.626+0000 7f553f7fe640 1 --2- 172.21.15.19:0/3027487759 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f554406c210 0x7f554406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.629 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.626+0000 7f55567fc640 1 --2- 172.21.15.19:0/3027487759 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f554406c210 0x7f554406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:34.629 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.626+0000 7f55567fc640 10 monclient: get_auth_request con 0x7f554406c210 auth_method 0 2023-10-06T14:26:34.629 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.627+0000 7f55567fc640 1 --2- 172.21.15.19:0/3027487759 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f554406c210 0x7f554406e6c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5558144f10 tx=0x7f554c007930 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.627+0000 7f553f7fe640 10 client.17198.objecter ms_handle_connect 0x7f554406c210 2023-10-06T14:26:34.643 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.640+0000 7f553f7fe640 1 -- 172.21.15.19:0/3027487759 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(970..970 src has 714..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f554804b2c0 con 0x7f55580b00c0 2023-10-06T14:26:34.643 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.640+0000 7f553f7fe640 10 client.17198.objecter ms_dispatch 0x7f5558000b90 osd_map(970..970 src has 714..970) v4 2023-10-06T14:26:34.643 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.640+0000 7f553f7fe640 3 client.17198.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:34.643 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.640+0000 7f553f7fe640 3 client.17198.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:34.643 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.640+0000 7f553f7fe640 20 client.17198.objecter dump_active .. 0 homeless 2023-10-06T14:26:34.643 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.641+0000 7f553f7fe640 1 -- 172.21.15.19:0/3027487759 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f5548075ff0 con 0x7f55580b00c0 2023-10-06T14:26:34.643 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.641+0000 7f553f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:34.643 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.641+0000 7f553f7fe640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:26:34.824 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.822+0000 7f555d531640 1 -- 172.21.15.19:0/3027487759 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f55580af3f0 con 0x7f554406c210 2023-10-06T14:26:34.825 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.823+0000 7f553f7fe640 1 -- 172.21.15.19:0/3027487759 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f554c0058b0 con 0x7f554406c210 2023-10-06T14:26:34.825 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T14:26:34.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 10 librados: watch_flush enter 2023-10-06T14:26:34.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 10 librados: watch_flush exit 2023-10-06T14:26:34.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 20 client.17198.objecter shutdown clearing up homeless session... 2023-10-06T14:26:34.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 10 client.17198.objecter successfully canceled tick 2023-10-06T14:26:34.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 1 -- 172.21.15.19:0/3027487759 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f554406c210 msgr2=0x7f554406e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:34.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 1 --2- 172.21.15.19:0/3027487759 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f554406c210 0x7f554406e6c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5558144f10 tx=0x7f554c007930 comp rx=0 tx=0).stop 2023-10-06T14:26:34.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 10 monclient: shutdown 2023-10-06T14:26:34.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:34.828 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 1 -- 172.21.15.19:0/3027487759 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f55580b00c0 msgr2=0x7f5558144610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:34.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 1 --2- 172.21.15.19:0/3027487759 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f55580b00c0 0x7f5558144610 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 crypto rx=0x7f5548000c00 tx=0x7f55480046c0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 1 -- 172.21.15.19:0/3027487759 shutdown_connections 2023-10-06T14:26:34.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 1 --2- 172.21.15.19:0/3027487759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55580b3d60 0x7f55581d67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 1 --2- 172.21.15.19:0/3027487759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55580b3990 0x7f55580ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 1 --2- 172.21.15.19:0/3027487759 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f554406c210 0x7f554406e6c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 1 --2- 172.21.15.19:0/3027487759 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f55580b00c0 0x7f5558144610 unknown :-1 s=CLOSED pgs=3132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 1 -- 172.21.15.19:0/3027487759 >> 172.21.15.19:0/3027487759 conn(0x7f55580b6e00 msgr2=0x7f555806b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:34.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 1 -- 172.21.15.19:0/3027487759 shutdown_connections 2023-10-06T14:26:34.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 1 -- 172.21.15.19:0/3027487759 wait complete. 2023-10-06T14:26:34.829 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.825+0000 7f555d531640 1 librados: shutdown 2023-10-06T14:26:34.838 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T14:26:34.838 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T14:26:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.948+0000 7fe941ce2640 10 monclient: get_monmap_and_config 2023-10-06T14:26:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.948+0000 7fe941ce2640 10 monclient: build_initial_monmap 2023-10-06T14:26:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.948+0000 7fe941ce2640 10 monclient: monmap: 2023-10-06T14:26:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:34.949480+0000 2023-10-06T14:26:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:34.949480+0000 2023-10-06T14:26:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:34.952 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 1 Processor -- start 2023-10-06T14:26:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 1 -- start start 2023-10-06T14:26:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 10 monclient: init 2023-10-06T14:26:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 10 monclient: _add_conns ranks=[3,4,2,0,1] 2023-10-06T14:26:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe93c0b3990 0x7fe93c0af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 10 monclient(hunting): picked mon.noname-a con 0x7fe93c0b3990 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe93c0b3d60 0x7fe93c141500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 10 monclient(hunting): picked mon.noname-c con 0x7fe93c0b3d60 addr v2:172.21.15.83:3300/0 2023-10-06T14:26:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe93c141a40 0x7fe93c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 10 monclient(hunting): picked mon.noname-b con 0x7fe93c141a40 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe93c140dd0 con 0x7fe93c0b3990 2023-10-06T14:26:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe93c0aebe0 con 0x7fe93c0b3d60 2023-10-06T14:26:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c141070 con 0x7fe93c141a40 2023-10-06T14:26:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.949+0000 7fe941ce2640 10 monclient(hunting): authenticate will time out at 3323.136203s 2023-10-06T14:26:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93bfff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe93c141a40 0x7fe93c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe93c0b3d60 0x7fe93c141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe93c0b3990 0x7fe93c0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93bfff640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe93c141a40 0x7fe93c0b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:55454/0 (socket says 172.21.15.19:55454) 2023-10-06T14:26:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93bfff640 1 -- 172.21.15.19:0/3928464939 learned_addr learned my addr 172.21.15.19:0/3928464939 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93bfff640 10 monclient(hunting): get_auth_request con 0x7fe93c141a40 auth_method 0 2023-10-06T14:26:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93bfff640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93bfff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93b7fe640 10 monclient(hunting): get_auth_request con 0x7fe93c0b3990 auth_method 0 2023-10-06T14:26:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93b7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93affd640 10 monclient(hunting): get_auth_request con 0x7fe93c0b3d60 auth_method 0 2023-10-06T14:26:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93affd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.950+0000 7fe93affd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.951+0000 7fe93bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.951+0000 7fe93bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.951+0000 7fe93bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.951+0000 7fe93affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.951+0000 7fe93affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.951+0000 7fe93affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.951+0000 7fe93b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.951+0000 7fe93b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.951+0000 7fe93b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.953+0000 7fe93affd640 10 monclient(hunting): handle_auth_done global_id 20809 payload 274 2023-10-06T14:26:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.953+0000 7fe93affd640 1 -- 172.21.15.19:0/3928464939 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe93c141a40 msgr2=0x7fe93c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.953+0000 7fe93affd640 1 --2- 172.21.15.19:0/3928464939 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe93c141a40 0x7fe93c0b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.953+0000 7fe93affd640 1 -- 172.21.15.19:0/3928464939 >> v2:172.21.15.19:3300/0 conn(0x7fe93c0b3990 msgr2=0x7fe93c0af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.953+0000 7fe93affd640 1 --2- 172.21.15.19:0/3928464939 >> v2:172.21.15.19:3300/0 conn(0x7fe93c0b3990 0x7fe93c0af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.953+0000 7fe93affd640 10 monclient: _finish_hunting 0 2023-10-06T14:26:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.953+0000 7fe93affd640 1 monclient: found mon.noname-c 2023-10-06T14:26:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.953+0000 7fe93affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.953+0000 7fe93affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T14:26:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.953+0000 7fe93affd640 1 -- 172.21.15.19:0/3928464939 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c0aee30 con 0x7fe93c0b3d60 2023-10-06T14:26:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.955+0000 7fe93affd640 1 --2- 172.21.15.19:0/3928464939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe93c0b3d60 0x7fe93c141500 secure :-1 s=READY pgs=5125 cs=0 l=1 rev1=1 crypto rx=0x7fe93c06a5f0 tx=0x7fe92402b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5639057e1a4c6f76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe93a7fc640 1 -- 172.21.15.19:0/3928464939 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92402f050 con 0x7fe93c0b3d60 2023-10-06T14:26:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe93a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe93a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T14:26:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe93a7fc640 10 monclient: dump: 2023-10-06T14:26:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:34.962 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe93a7fc640 10 monclient: _finish_auth 0 2023-10-06T14:26:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe93a7fc640 10 monclient: _check_auth_tickets 2023-10-06T14:26:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe93a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe93a7fc640 1 -- 172.21.15.19:0/3928464939 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe924031700 con 0x7fe93c0b3d60 2023-10-06T14:26:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe93a7fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe93a7fc640 1 -- 172.21.15.19:0/3928464939 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924031a90 con 0x7fe93c0b3d60 2023-10-06T14:26:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe941ce2640 5 monclient: authenticate success, global_id 20809 2023-10-06T14:26:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe941ce2640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe941ce2640 4 set_mon_vals no callback set 2023-10-06T14:26:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe93a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe93a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.958+0000 7fe93a7fc640 10 monclient: dump: 2023-10-06T14:26:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.964 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.964 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:34.964 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:34.964 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:34.964 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:34.964 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:34.964 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:34.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.959+0000 7fe941ce2640 10 monclient: shutdown 2023-10-06T14:26:34.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.959+0000 7fe941ce2640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:34.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.959+0000 7fe941ce2640 1 -- 172.21.15.19:0/3928464939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe93c0b3d60 msgr2=0x7fe93c141500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:34.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.959+0000 7fe941ce2640 1 --2- 172.21.15.19:0/3928464939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe93c0b3d60 0x7fe93c141500 secure :-1 s=READY pgs=5125 cs=0 l=1 rev1=1 crypto rx=0x7fe93c06a5f0 tx=0x7fe92402b7a0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.964 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.959+0000 7fe941ce2640 1 -- 172.21.15.19:0/3928464939 shutdown_connections 2023-10-06T14:26:34.965 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.959+0000 7fe941ce2640 1 --2- 172.21.15.19:0/3928464939 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe93c141a40 0x7fe93c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.965 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.959+0000 7fe941ce2640 1 --2- 172.21.15.19:0/3928464939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe93c0b3d60 0x7fe93c141500 unknown :-1 s=CLOSED pgs=5125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.965 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.959+0000 7fe941ce2640 1 --2- 172.21.15.19:0/3928464939 >> v2:172.21.15.19:3300/0 conn(0x7fe93c0b3990 0x7fe93c0af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.965 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.959+0000 7fe941ce2640 1 -- 172.21.15.19:0/3928464939 >> 172.21.15.19:0/3928464939 conn(0x7fe93c0b6e00 msgr2=0x7fe93c0b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:34.965 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.959+0000 7fe941ce2640 1 -- 172.21.15.19:0/3928464939 shutdown_connections 2023-10-06T14:26:34.965 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.959+0000 7fe941ce2640 1 -- 172.21.15.19:0/3928464939 wait complete. 2023-10-06T14:26:34.968 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.960+0000 7fe941ce2640 10 monclient: build_initial_monmap 2023-10-06T14:26:34.968 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.960+0000 7fe941ce2640 10 monclient: monmap: 2023-10-06T14:26:34.968 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:34.968 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:34.969 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:34.969 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:34.969 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:34.969 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:34.969 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:34.969 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:34.969 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:34.969 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:34.969 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.960+0000 7fe941ce2640 1 librados: starting msgr at 2023-10-06T14:26:34.969 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.960+0000 7fe941ce2640 1 librados: starting objecter 2023-10-06T14:26:34.969 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.960+0000 7fe941ce2640 1 Processor -- start 2023-10-06T14:26:34.969 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.960+0000 7fe941ce2640 1 -- start start 2023-10-06T14:26:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.960+0000 7fe941ce2640 1 librados: setting wanted keys 2023-10-06T14:26:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.960+0000 7fe941ce2640 1 librados: calling monclient init 2023-10-06T14:26:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.960+0000 7fe941ce2640 10 monclient: init 2023-10-06T14:26:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T14:26:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe93c0b3990 0x7fe93c1d9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 10 monclient(hunting): picked mon.noname-b con 0x7fe93c0b3990 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe93c0b3d60 0x7fe93c13cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 10 monclient(hunting): picked mon.noname-c con 0x7fe93c0b3d60 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c141a40 0x7fe93c13d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 10 monclient(hunting): picked mon.noname-a con 0x7fe93c141a40 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c13da60 con 0x7fe93c141a40 2023-10-06T14:26:34.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:34.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c0a8650 con 0x7fe93c0b3990 2023-10-06T14:26:34.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:34.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c1d9eb0 con 0x7fe93c0b3d60 2023-10-06T14:26:34.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:34.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe941ce2640 10 monclient(hunting): authenticate will time out at 3323.147606s 2023-10-06T14:26:34.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe93affd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe93c0b3d60 0x7fe93c13cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:34.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe93b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe93c0b3990 0x7fe93c1d9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:34.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe93bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c141a40 0x7fe93c13d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:34.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.961+0000 7fe93affd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe93c0b3d60 0x7fe93c13cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:55464/0 (socket says 172.21.15.19:55464) 2023-10-06T14:26:34.971 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93affd640 1 -- 172.21.15.19:0/3073810646 learned_addr learned my addr 172.21.15.19:0/3073810646 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93affd640 10 monclient(hunting): get_auth_request con 0x7fe93c0b3d60 auth_method 0 2023-10-06T14:26:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93affd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93affd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93b7fe640 10 monclient(hunting): get_auth_request con 0x7fe93c0b3990 auth_method 0 2023-10-06T14:26:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93b7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93bfff640 10 monclient(hunting): get_auth_request con 0x7fe93c141a40 auth_method 0 2023-10-06T14:26:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93bfff640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93bfff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:34.973 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:34.973 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:34.973 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:34.973 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:34.973 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:34.973 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.962+0000 7fe93bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.965+0000 7fe93b7fe640 10 monclient(hunting): handle_auth_done global_id 20812 payload 824 2023-10-06T14:26:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.965+0000 7fe93b7fe640 1 -- 172.21.15.19:0/3073810646 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe93c0b3d60 msgr2=0x7fe93c13cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.965+0000 7fe93b7fe640 1 --2- 172.21.15.19:0/3073810646 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fe93c0b3d60 0x7fe93c13cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.965+0000 7fe93b7fe640 1 -- 172.21.15.19:0/3073810646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c141a40 msgr2=0x7fe93c13d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.965+0000 7fe93b7fe640 1 --2- 172.21.15.19:0/3073810646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c141a40 0x7fe93c13d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.965+0000 7fe93b7fe640 10 monclient: _finish_hunting 0 2023-10-06T14:26:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.965+0000 7fe93b7fe640 1 monclient: found mon.noname-b 2023-10-06T14:26:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.965+0000 7fe93b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.965+0000 7fe93b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T14:26:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.965+0000 7fe93b7fe640 1 -- 172.21.15.19:0/3073810646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe924009400 con 0x7fe93c0b3990 2023-10-06T14:26:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.967+0000 7fe93b7fe640 1 --2- 172.21.15.19:0/3073810646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe93c0b3990 0x7fe93c1d9450 secure :-1 s=READY pgs=5126 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00e870 tx=0x7fe92c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.967+0000 7fe933fff640 10 client.?.objecter ms_handle_connect 0x7fe93c0b3990 2023-10-06T14:26:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.967+0000 7fe933fff640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe933fff640 1 -- 172.21.15.19:0/3073810646 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00c940 con 0x7fe93c0b3990 2023-10-06T14:26:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe933fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe933fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe933fff640 10 monclient: dump: 2023-10-06T14:26:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:34.977 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe933fff640 10 monclient: _finish_auth 0 2023-10-06T14:26:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe933fff640 10 monclient: _check_auth_tickets 2023-10-06T14:26:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe933fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe933fff640 1 -- 172.21.15.19:0/3073810646 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92c00caa0 con 0x7fe93c0b3990 2023-10-06T14:26:34.978 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe933fff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:34.978 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe933fff640 1 -- 172.21.15.19:0/3073810646 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00ce10 con 0x7fe93c0b3990 2023-10-06T14:26:34.978 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe933fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:34.978 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe933fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:34.978 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe933fff640 10 monclient: dump: 2023-10-06T14:26:34.978 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:34.978 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:34.978 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.978 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:34.979 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe941ce2640 5 monclient: authenticate success, global_id 20812 2023-10-06T14:26:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe941ce2640 10 monclient: _renew_subs 2023-10-06T14:26:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe941ce2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe939ffb640 4 set_mon_vals no callback set 2023-10-06T14:26:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe941ce2640 1 -- 172.21.15.19:0/3073810646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93c1d9990 con 0x7fe93c0b3990 2023-10-06T14:26:34.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe941ce2640 10 client.20812.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:34.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe941ce2640 10 monclient: _renew_subs 2023-10-06T14:26:34.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe941ce2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:34.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe941ce2640 1 -- 172.21.15.19:0/3073810646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93c0acdf0 con 0x7fe93c0b3990 2023-10-06T14:26:34.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.970+0000 7fe941ce2640 1 librados: init done 2023-10-06T14:26:34.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.971+0000 7fe941ce2640 10 client.20812.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:34.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.971+0000 7fe941ce2640 15 client.20812.objecter _session_command_op_assign -1 1 2023-10-06T14:26:34.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.971+0000 7fe941ce2640 20 client.20812.objecter _get_session osd=-1 returning homeless 2023-10-06T14:26:34.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.971+0000 7fe941ce2640 10 client.20812.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:34.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.971+0000 7fe941ce2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:34.980 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.971+0000 7fe941ce2640 1 -- 172.21.15.19:0/3073810646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe93c0af3f0 con 0x7fe93c0b3990 2023-10-06T14:26:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.989+0000 7fe933fff640 1 -- 172.21.15.19:0/3073810646 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe92c013470 con 0x7fe93c0b3990 2023-10-06T14:26:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.989+0000 7fe933fff640 1 --2- 172.21.15.19:0/3073810646 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe91c06c210 0x7fe91c06e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.989+0000 7fe93affd640 1 --2- 172.21.15.19:0/3073810646 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe91c06c210 0x7fe91c06e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.989+0000 7fe93affd640 10 monclient: get_auth_request con 0x7fe91c06c210 auth_method 0 2023-10-06T14:26:34.992 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.990+0000 7fe93affd640 1 --2- 172.21.15.19:0/3073810646 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe91c06c210 0x7fe91c06e6c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe93c1cc220 tx=0x7fe924008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:34.992 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.990+0000 7fe933fff640 10 client.20812.objecter ms_handle_connect 0x7fe91c06c210 2023-10-06T14:26:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 1 -- 172.21.15.19:0/3073810646 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(970..970 src has 714..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fe92c058330 con 0x7fe93c0b3990 2023-10-06T14:26:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 10 client.20812.objecter ms_dispatch 0x7fe93c000b90 osd_map(970..970 src has 714..970) v4 2023-10-06T14:26:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 3 client.20812.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 3 client.20812.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 10 client.20812.objecter checking command 1 2023-10-06T14:26:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 1 --2- 172.21.15.19:0/3073810646 >> [v2:172.21.15.83:6810/86125,v1:172.21.15.83:6811/86125] conn(0x7fe91c0739a0 0x7fe91c075e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 20 client.20812.objecter _get_session s=0x7fe91c0732d0 osd=7 3 2023-10-06T14:26:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 20 client.20812.objecter put_session s=0x7fe91c0732d0 osd=7 3 2023-10-06T14:26:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 15 client.20812.objecter _session_command_op_remove -1 1 2023-10-06T14:26:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 20 client.20812.objecter _get_session s=0x7fe91c0732d0 osd=7 3 2023-10-06T14:26:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 20 client.20812.objecter get_session s=0x7fe91c0732d0 osd=7 3 2023-10-06T14:26:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 15 client.20812.objecter _session_command_op_assign 7 1 2023-10-06T14:26:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 20 client.20812.objecter put_session s=0x7fe91c0732d0 osd=7 4 2023-10-06T14:26:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 10 client.20812.objecter _send_command 1 2023-10-06T14:26:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 1 -- 172.21.15.19:0/3073810646 --> [v2:172.21.15.83:6810/86125,v1:172.21.15.83:6811/86125] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe91c076500 con 0x7fe91c0739a0 2023-10-06T14:26:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.991+0000 7fe933fff640 20 client.20812.objecter dump_active .. 0 homeless 2023-10-06T14:26:34.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.992+0000 7fe93bfff640 1 --2- 172.21.15.19:0/3073810646 >> [v2:172.21.15.83:6810/86125,v1:172.21.15.83:6811/86125] conn(0x7fe91c0739a0 0x7fe91c075e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:34.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.992+0000 7fe93bfff640 10 monclient: get_auth_request con 0x7fe91c0739a0 auth_method 0 2023-10-06T14:26:34.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.993+0000 7fe933fff640 1 -- 172.21.15.19:0/3073810646 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=970) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe92c054670 con 0x7fe93c0b3990 2023-10-06T14:26:34.995 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.993+0000 7fe933fff640 10 monclient: handle_get_version_reply finishing 1 version 970 2023-10-06T14:26:34.997 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.995+0000 7fe933fff640 10 client.20812.objecter ms_handle_connect 0x7fe91c0739a0 2023-10-06T14:26:34.997 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:34.995+0000 7fe93bfff640 1 --2- 172.21.15.19:0/3073810646 >> [v2:172.21.15.83:6810/86125,v1:172.21.15.83:6811/86125] conn(0x7fe91c0739a0 0x7fe91c075e30 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:35.018 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.016+0000 7fe933fff640 1 -- 172.21.15.19:0/3073810646 <== osd.7 v2:172.21.15.83:6810/86125 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe91c076500 con 0x7fe91c0739a0 2023-10-06T14:26:35.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.016+0000 7fe933fff640 10 client.20812.objecter ms_dispatch 0x7fe93c000b90 command_reply(tid 1: 0 ) v1 2023-10-06T14:26:35.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.016+0000 7fe933fff640 10 client.20812.objecter _finish_command 1 = system:0 2023-10-06T14:26:35.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.017+0000 7fe933fff640 20 client.20812.objecter put_session s=0x7fe91c0732d0 osd=7 4 2023-10-06T14:26:35.019 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.017+0000 7fe933fff640 15 client.20812.objecter _session_command_op_remove 7 1 2023-10-06T14:26:35.031 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.029+0000 7fe941ce2640 10 client.20812.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-10-06T14:26:35.031 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.029+0000 7fe941ce2640 15 client.20812.objecter _session_command_op_assign -1 2 2023-10-06T14:26:35.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.029+0000 7fe941ce2640 20 client.20812.objecter _get_session s=0x7fe91c0732d0 osd=7 3 2023-10-06T14:26:35.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.029+0000 7fe941ce2640 20 client.20812.objecter put_session s=0x7fe91c0732d0 osd=7 3 2023-10-06T14:26:35.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.029+0000 7fe941ce2640 20 client.20812.objecter _get_session s=0x7fe91c0732d0 osd=7 3 2023-10-06T14:26:35.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.029+0000 7fe941ce2640 15 client.20812.objecter _session_command_op_remove -1 2 2023-10-06T14:26:35.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.029+0000 7fe941ce2640 20 client.20812.objecter get_session s=0x7fe91c0732d0 osd=7 3 2023-10-06T14:26:35.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.029+0000 7fe941ce2640 15 client.20812.objecter _session_command_op_assign 7 2 2023-10-06T14:26:35.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.029+0000 7fe941ce2640 20 client.20812.objecter put_session s=0x7fe91c0732d0 osd=7 4 2023-10-06T14:26:35.032 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.029+0000 7fe941ce2640 10 client.20812.objecter _send_command 2 2023-10-06T14:26:35.033 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.029+0000 7fe941ce2640 1 -- 172.21.15.19:0/3073810646 --> [v2:172.21.15.83:6810/86125,v1:172.21.15.83:6811/86125] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe93c1da070 con 0x7fe91c0739a0 2023-10-06T14:26:35.046 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.043+0000 7fe933fff640 1 -- 172.21.15.19:0/3073810646 <== osd.7 v2:172.21.15.83:6810/86125 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe93c1da070 con 0x7fe91c0739a0 2023-10-06T14:26:35.046 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.043+0000 7fe933fff640 10 client.20812.objecter ms_dispatch 0x7fe93c000b90 command_reply(tid 2: 0 ) v1 2023-10-06T14:26:35.046 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.043+0000 7fe933fff640 10 client.20812.objecter _finish_command 2 = system:0 2023-10-06T14:26:35.046 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.043+0000 7fe933fff640 20 client.20812.objecter put_session s=0x7fe91c0732d0 osd=7 4 2023-10-06T14:26:35.046 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.043+0000 7fe933fff640 15 client.20812.objecter _session_command_op_remove 7 2 2023-10-06T14:26:35.046 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2023-10-06T14:26:35.047 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T14:26:35.047 INFO:teuthology.orchestra.run.smithi019.stdout:} 2023-10-06T14:26:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 10 librados: watch_flush enter 2023-10-06T14:26:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 10 librados: watch_flush exit 2023-10-06T14:26:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 10 client.20812.objecter close_session for osd.7 2023-10-06T14:26:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 1 -- 172.21.15.19:0/3073810646 >> [v2:172.21.15.83:6810/86125,v1:172.21.15.83:6811/86125] conn(0x7fe91c0739a0 msgr2=0x7fe91c075e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 1 --2- 172.21.15.19:0/3073810646 >> [v2:172.21.15.83:6810/86125,v1:172.21.15.83:6811/86125] conn(0x7fe91c0739a0 0x7fe91c075e30 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 20 client.20812.objecter put_session s=0x7fe91c0732d0 osd=7 1 2023-10-06T14:26:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 20 client.20812.objecter shutdown clearing up homeless session... 2023-10-06T14:26:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 10 client.20812.objecter successfully canceled tick 2023-10-06T14:26:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 1 -- 172.21.15.19:0/3073810646 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe91c06c210 msgr2=0x7fe91c06e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 1 --2- 172.21.15.19:0/3073810646 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7fe91c06c210 0x7fe91c06e6c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe93c1cc220 tx=0x7fe924008030 comp rx=0 tx=0).stop 2023-10-06T14:26:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 10 monclient: shutdown 2023-10-06T14:26:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 1 -- 172.21.15.19:0/3073810646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe93c0b3990 msgr2=0x7fe93c1d9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 1 --2- 172.21.15.19:0/3073810646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe93c0b3990 0x7fe93c1d9450 secure :-1 s=READY pgs=5126 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00e870 tx=0x7fe92c00ed30 comp rx=0 tx=0).stop 2023-10-06T14:26:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe93bfff640 1 -- 172.21.15.19:0/3073810646 reap_dead start 2023-10-06T14:26:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 1 -- 172.21.15.19:0/3073810646 shutdown_connections 2023-10-06T14:26:35.050 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 1 -- 172.21.15.19:0/3073810646 >> 172.21.15.19:0/3073810646 conn(0x7fe93c0b6e00 msgr2=0x7fe93c0b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:35.050 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 1 -- 172.21.15.19:0/3073810646 shutdown_connections 2023-10-06T14:26:35.050 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 1 -- 172.21.15.19:0/3073810646 wait complete. 2023-10-06T14:26:35.050 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.044+0000 7fe941ce2640 1 librados: shutdown 2023-10-06T14:26:35.056 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T14:26:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.165+0000 7ff5dd421640 10 monclient: get_monmap_and_config 2023-10-06T14:26:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.165+0000 7ff5dd421640 10 monclient: build_initial_monmap 2023-10-06T14:26:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.165+0000 7ff5dd421640 10 monclient: monmap: 2023-10-06T14:26:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:35.166173+0000 2023-10-06T14:26:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:35.166173+0000 2023-10-06T14:26:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:35.168 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.165+0000 7ff5dd421640 1 Processor -- start 2023-10-06T14:26:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.165+0000 7ff5dd421640 1 -- start start 2023-10-06T14:26:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.165+0000 7ff5dd421640 10 monclient: init 2023-10-06T14:26:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2023-10-06T14:26:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff5d80b3990 0x7ff5d80af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 10 monclient(hunting): picked mon.noname-b con 0x7ff5d80b3990 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff5d80b3d60 0x7ff5d8141500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 10 monclient(hunting): picked mon.noname-c con 0x7ff5d80b3d60 addr v2:172.21.15.83:3300/0 2023-10-06T14:26:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff5d8141a40 0x7ff5d80b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 10 monclient(hunting): picked mon.noname-a con 0x7ff5d8141a40 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d8140dd0 con 0x7ff5d8141a40 2023-10-06T14:26:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d80aebe0 con 0x7ff5d80b3d60 2023-10-06T14:26:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d8141070 con 0x7ff5d80b3990 2023-10-06T14:26:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5dd421640 10 monclient(hunting): authenticate will time out at 3323.352777s 2023-10-06T14:26:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5d6ffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff5d80b3990 0x7ff5d80af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.166+0000 7ff5d67fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff5d80b3d60 0x7ff5d8141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d6ffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff5d80b3990 0x7ff5d80af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:55478/0 (socket says 172.21.15.19:55478) 2023-10-06T14:26:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d6ffd640 1 -- 172.21.15.19:0/353415226 learned_addr learned my addr 172.21.15.19:0/353415226 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d6ffd640 10 monclient(hunting): get_auth_request con 0x7ff5d80b3990 auth_method 0 2023-10-06T14:26:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d6ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d6ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d67fc640 10 monclient(hunting): get_auth_request con 0x7ff5d80b3d60 auth_method 0 2023-10-06T14:26:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d67fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d67fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.167+0000 7ff5d67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:35.174 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.169+0000 7ff5d67fc640 10 monclient(hunting): handle_auth_done global_id 20815 payload 274 2023-10-06T14:26:35.174 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.169+0000 7ff5d67fc640 1 -- 172.21.15.19:0/353415226 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff5d80b3990 msgr2=0x7ff5d80af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:35.174 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.169+0000 7ff5d67fc640 1 --2- 172.21.15.19:0/353415226 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff5d80b3990 0x7ff5d80af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.174 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.169+0000 7ff5d67fc640 1 -- 172.21.15.19:0/353415226 >> v2:172.21.15.19:3300/0 conn(0x7ff5d8141a40 msgr2=0x7ff5d80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:35.174 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.169+0000 7ff5d67fc640 1 --2- 172.21.15.19:0/353415226 >> v2:172.21.15.19:3300/0 conn(0x7ff5d8141a40 0x7ff5d80b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.174 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.169+0000 7ff5d67fc640 10 monclient: _finish_hunting 0 2023-10-06T14:26:35.174 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.169+0000 7ff5d67fc640 1 monclient: found mon.noname-c 2023-10-06T14:26:35.175 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.169+0000 7ff5d67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:35.175 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.169+0000 7ff5d67fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T14:26:35.175 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.169+0000 7ff5d67fc640 1 -- 172.21.15.19:0/353415226 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d80aee30 con 0x7ff5d80b3d60 2023-10-06T14:26:35.175 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.171+0000 7ff5d67fc640 1 --2- 172.21.15.19:0/353415226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5d80b3d60 0x7ff5d8141500 secure :-1 s=READY pgs=5127 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8072400 tx=0x7ff5c002b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9986fa247307652 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:35.176 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5d5ffb640 1 -- 172.21.15.19:0/353415226 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c002f050 con 0x7ff5d80b3d60 2023-10-06T14:26:35.176 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5d5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T14:26:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5d5ffb640 10 monclient: dump: 2023-10-06T14:26:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:35.178 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5d5ffb640 10 monclient: _finish_auth 0 2023-10-06T14:26:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5d5ffb640 10 monclient: _check_auth_tickets 2023-10-06T14:26:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5d5ffb640 1 -- 172.21.15.19:0/353415226 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c0031700 con 0x7ff5d80b3d60 2023-10-06T14:26:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5dd421640 5 monclient: authenticate success, global_id 20815 2023-10-06T14:26:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5dd421640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5d5ffb640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5dd421640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5d5ffb640 1 -- 172.21.15.19:0/353415226 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0031a90 con 0x7ff5d80b3d60 2023-10-06T14:26:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5dd421640 4 set_mon_vals no callback set 2023-10-06T14:26:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5d5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.174+0000 7ff5d5ffb640 10 monclient: dump: 2023-10-06T14:26:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:35.180 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:35.182 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.175+0000 7ff5dd421640 10 monclient: shutdown 2023-10-06T14:26:35.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.175+0000 7ff5dd421640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:35.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.175+0000 7ff5dd421640 1 -- 172.21.15.19:0/353415226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5d80b3d60 msgr2=0x7ff5d8141500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:35.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.175+0000 7ff5dd421640 1 --2- 172.21.15.19:0/353415226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5d80b3d60 0x7ff5d8141500 secure :-1 s=READY pgs=5127 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8072400 tx=0x7ff5c002b7a0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.175+0000 7ff5dd421640 1 -- 172.21.15.19:0/353415226 shutdown_connections 2023-10-06T14:26:35.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.175+0000 7ff5dd421640 1 --2- 172.21.15.19:0/353415226 >> v2:172.21.15.19:3300/0 conn(0x7ff5d8141a40 0x7ff5d80b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.175+0000 7ff5dd421640 1 --2- 172.21.15.19:0/353415226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5d80b3d60 0x7ff5d8141500 unknown :-1 s=CLOSED pgs=5127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.175+0000 7ff5dd421640 1 --2- 172.21.15.19:0/353415226 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff5d80b3990 0x7ff5d80af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.175+0000 7ff5dd421640 1 -- 172.21.15.19:0/353415226 >> 172.21.15.19:0/353415226 conn(0x7ff5d80b6e00 msgr2=0x7ff5d80b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:35.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.175+0000 7ff5dd421640 1 -- 172.21.15.19:0/353415226 shutdown_connections 2023-10-06T14:26:35.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.175+0000 7ff5dd421640 1 -- 172.21.15.19:0/353415226 wait complete. 2023-10-06T14:26:35.183 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.176+0000 7ff5dd421640 10 monclient: build_initial_monmap 2023-10-06T14:26:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.176+0000 7ff5dd421640 10 monclient: monmap: 2023-10-06T14:26:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:35.184 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.176+0000 7ff5dd421640 1 librados: starting msgr at 2023-10-06T14:26:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.176+0000 7ff5dd421640 1 librados: starting objecter 2023-10-06T14:26:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.176+0000 7ff5dd421640 1 Processor -- start 2023-10-06T14:26:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.176+0000 7ff5dd421640 1 -- start start 2023-10-06T14:26:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.176+0000 7ff5dd421640 1 librados: setting wanted keys 2023-10-06T14:26:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.176+0000 7ff5dd421640 1 librados: calling monclient init 2023-10-06T14:26:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.176+0000 7ff5dd421640 10 monclient: init 2023-10-06T14:26:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 10 monclient: _add_conns ranks=[1,0,2] 2023-10-06T14:26:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5d80b3990 0x7ff5d80ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 10 monclient(hunting): picked mon.noname-b con 0x7ff5d80b3990 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5d80b3d60 0x7ff5d81d9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 10 monclient(hunting): picked mon.noname-a con 0x7ff5d80b3d60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff5d8141a40 0x7ff5d81d9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 10 monclient(hunting): picked mon.noname-c con 0x7ff5d8141a40 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d81c6f70 con 0x7ff5d80b3d60 2023-10-06T14:26:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d813bca0 con 0x7ff5d80b3990 2023-10-06T14:26:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d81c73c0 con 0x7ff5d8141a40 2023-10-06T14:26:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5dd421640 10 monclient(hunting): authenticate will time out at 3323.363851s 2023-10-06T14:26:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.177+0000 7ff5d77fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff5d8141a40 0x7ff5d81d9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.178+0000 7ff5d6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5d80b3990 0x7ff5d80ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.178+0000 7ff5d77fe640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff5d8141a40 0x7ff5d81d9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:55492/0 (socket says 172.21.15.19:55492) 2023-10-06T14:26:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.178+0000 7ff5d77fe640 1 -- 172.21.15.19:0/331372332 learned_addr learned my addr 172.21.15.19:0/331372332 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.178+0000 7ff5d77fe640 10 monclient(hunting): get_auth_request con 0x7ff5d8141a40 auth_method 0 2023-10-06T14:26:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.178+0000 7ff5d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.178+0000 7ff5d77fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.178+0000 7ff5d77fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.178+0000 7ff5d6ffd640 10 monclient(hunting): get_auth_request con 0x7ff5d80b3990 auth_method 0 2023-10-06T14:26:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.178+0000 7ff5d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.178+0000 7ff5d6ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.178+0000 7ff5d6ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.178+0000 7ff5d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.178+0000 7ff5d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.178+0000 7ff5d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.179+0000 7ff5d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.179+0000 7ff5d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.179+0000 7ff5d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.181+0000 7ff5d6ffd640 10 monclient(hunting): handle_auth_done global_id 20818 payload 824 2023-10-06T14:26:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.181+0000 7ff5d6ffd640 1 -- 172.21.15.19:0/331372332 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff5d8141a40 msgr2=0x7ff5d81d9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.181+0000 7ff5d6ffd640 1 --2- 172.21.15.19:0/331372332 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff5d8141a40 0x7ff5d81d9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.181+0000 7ff5d6ffd640 1 -- 172.21.15.19:0/331372332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5d80b3d60 msgr2=0x7ff5d81d9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.181+0000 7ff5d6ffd640 1 --2- 172.21.15.19:0/331372332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5d80b3d60 0x7ff5d81d9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.181+0000 7ff5d6ffd640 10 monclient: _finish_hunting 0 2023-10-06T14:26:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.181+0000 7ff5d6ffd640 1 monclient: found mon.noname-b 2023-10-06T14:26:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.181+0000 7ff5d6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.181+0000 7ff5d6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T14:26:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.181+0000 7ff5d6ffd640 1 -- 172.21.15.19:0/331372332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c0009400 con 0x7ff5d80b3990 2023-10-06T14:26:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.183+0000 7ff5d6ffd640 1 --2- 172.21.15.19:0/331372332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5d80b3990 0x7ff5d80ac640 secure :-1 s=READY pgs=5128 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800ebc0 tx=0x7ff5c800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.183+0000 7ff5cf7fe640 10 client.?.objecter ms_handle_connect 0x7ff5d80b3990 2023-10-06T14:26:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.183+0000 7ff5cf7fe640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.186+0000 7ff5cf7fe640 1 -- 172.21.15.19:0/331372332 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80092d0 con 0x7ff5d80b3990 2023-10-06T14:26:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.186+0000 7ff5cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.186+0000 7ff5cf7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.186+0000 7ff5cf7fe640 10 monclient: dump: 2023-10-06T14:26:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:35.192 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.186+0000 7ff5cf7fe640 10 monclient: _finish_auth 0 2023-10-06T14:26:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.186+0000 7ff5cf7fe640 10 monclient: _check_auth_tickets 2023-10-06T14:26:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.186+0000 7ff5cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5cf7fe640 1 -- 172.21.15.19:0/331372332 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c800f040 con 0x7ff5d80b3990 2023-10-06T14:26:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5cf7fe640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5cf7fe640 1 -- 172.21.15.19:0/331372332 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800cd60 con 0x7ff5d80b3990 2023-10-06T14:26:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5cf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5cf7fe640 10 monclient: dump: 2023-10-06T14:26:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:35.193 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5dd421640 5 monclient: authenticate success, global_id 20818 2023-10-06T14:26:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5dd421640 10 monclient: _renew_subs 2023-10-06T14:26:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5dd421640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5dd421640 1 -- 172.21.15.19:0/331372332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d81c75e0 con 0x7ff5d80b3990 2023-10-06T14:26:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5d57fa640 4 set_mon_vals no callback set 2023-10-06T14:26:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5dd421640 10 client.20818.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5dd421640 10 monclient: _renew_subs 2023-10-06T14:26:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5dd421640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5dd421640 1 -- 172.21.15.19:0/331372332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d81c7a70 con 0x7ff5d80b3990 2023-10-06T14:26:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5dd421640 1 librados: init done 2023-10-06T14:26:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5dd421640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5dd421640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:35.195 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5dd421640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:35.195 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.187+0000 7ff5dd421640 1 -- 172.21.15.19:0/331372332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d80678e0 con 0x7ff5d80b3990 2023-10-06T14:26:35.208 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.206+0000 7ff5cf7fe640 1 -- 172.21.15.19:0/331372332 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff5c8014480 con 0x7ff5d80b3990 2023-10-06T14:26:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.206+0000 7ff5cf7fe640 1 --2- 172.21.15.19:0/331372332 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ff5b806c210 0x7ff5b806e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.206+0000 7ff5d67fc640 1 --2- 172.21.15.19:0/331372332 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ff5b806c210 0x7ff5b806e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.206+0000 7ff5d67fc640 10 monclient: get_auth_request con 0x7ff5b806c210 auth_method 0 2023-10-06T14:26:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.207+0000 7ff5d67fc640 1 --2- 172.21.15.19:0/331372332 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ff5b806c210 0x7ff5b806e6c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0035000 tx=0x7ff5c0030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:35.209 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.207+0000 7ff5cf7fe640 10 client.20818.objecter ms_handle_connect 0x7ff5b806c210 2023-10-06T14:26:35.212 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.209+0000 7ff5cf7fe640 1 -- 172.21.15.19:0/331372332 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(970..970 src has 714..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7ff5c80596c0 con 0x7ff5d80b3990 2023-10-06T14:26:35.212 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.210+0000 7ff5cf7fe640 10 client.20818.objecter ms_dispatch 0x7ff5d8000b90 osd_map(970..970 src has 714..970) v4 2023-10-06T14:26:35.212 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.210+0000 7ff5cf7fe640 3 client.20818.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:35.212 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.210+0000 7ff5cf7fe640 3 client.20818.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:35.212 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.210+0000 7ff5cf7fe640 20 client.20818.objecter dump_active .. 0 homeless 2023-10-06T14:26:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.438+0000 7ff5cf7fe640 1 -- 172.21.15.19:0/331372332 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7ff5c8055300 con 0x7ff5d80b3990 2023-10-06T14:26:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.438+0000 7ff5cf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.438+0000 7ff5cf7fe640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:26:35.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.616+0000 7ff5dd421640 1 -- 172.21.15.19:0/331372332 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5d80af3f0 con 0x7ff5b806c210 2023-10-06T14:26:35.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.617+0000 7ff5cf7fe640 1 -- 172.21.15.19:0/331372332 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff5c00318f0 con 0x7ff5b806c210 2023-10-06T14:26:35.619 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T14:26:35.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.618+0000 7ff5dd421640 10 librados: watch_flush enter 2023-10-06T14:26:35.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.618+0000 7ff5dd421640 10 librados: watch_flush exit 2023-10-06T14:26:35.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.618+0000 7ff5dd421640 20 client.20818.objecter shutdown clearing up homeless session... 2023-10-06T14:26:35.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.618+0000 7ff5dd421640 10 client.20818.objecter successfully canceled tick 2023-10-06T14:26:35.621 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 1 -- 172.21.15.19:0/331372332 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ff5b806c210 msgr2=0x7ff5b806e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:35.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 1 --2- 172.21.15.19:0/331372332 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ff5b806c210 0x7ff5b806e6c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0035000 tx=0x7ff5c0030000 comp rx=0 tx=0).stop 2023-10-06T14:26:35.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 10 monclient: shutdown 2023-10-06T14:26:35.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:35.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 1 -- 172.21.15.19:0/331372332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5d80b3990 msgr2=0x7ff5d80ac640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:35.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 1 --2- 172.21.15.19:0/331372332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5d80b3990 0x7ff5d80ac640 secure :-1 s=READY pgs=5128 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800ebc0 tx=0x7ff5c800c490 comp rx=0 tx=0).stop 2023-10-06T14:26:35.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 1 -- 172.21.15.19:0/331372332 shutdown_connections 2023-10-06T14:26:35.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 1 --2- 172.21.15.19:0/331372332 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7ff5b806c210 0x7ff5b806e6c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 1 --2- 172.21.15.19:0/331372332 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7ff5d8141a40 0x7ff5d81d9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 1 --2- 172.21.15.19:0/331372332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5d80b3d60 0x7ff5d81d9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 1 --2- 172.21.15.19:0/331372332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5d80b3990 0x7ff5d80ac640 unknown :-1 s=CLOSED pgs=5128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.622 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 1 -- 172.21.15.19:0/331372332 >> 172.21.15.19:0/331372332 conn(0x7ff5d80b6e00 msgr2=0x7ff5d80b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:35.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 1 -- 172.21.15.19:0/331372332 shutdown_connections 2023-10-06T14:26:35.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 1 -- 172.21.15.19:0/331372332 wait complete. 2023-10-06T14:26:35.623 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.619+0000 7ff5dd421640 1 librados: shutdown 2023-10-06T14:26:35.632 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T14:26:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.735+0000 7f23dbcb6640 10 monclient: get_monmap_and_config 2023-10-06T14:26:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.735+0000 7f23dbcb6640 10 monclient: build_initial_monmap 2023-10-06T14:26:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.735+0000 7f23dbcb6640 10 monclient: monmap: 2023-10-06T14:26:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:35.736162+0000 2023-10-06T14:26:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:35.736162+0000 2023-10-06T14:26:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:35.738 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:35.738 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:35.738 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:35.738 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:35.738 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:35.738 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:35.738 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.735+0000 7f23dbcb6640 1 Processor -- start 2023-10-06T14:26:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.735+0000 7f23dbcb6640 1 -- start start 2023-10-06T14:26:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.735+0000 7f23dbcb6640 10 monclient: init 2023-10-06T14:26:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.736+0000 7f23dbcb6640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.736+0000 7f23dbcb6640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2023-10-06T14:26:35.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.736+0000 7f23dbcb6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f23d40b3990 addr v1:172.21.15.83:6789/0 2023-10-06T14:26:35.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.736+0000 7f23dbcb6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f23d40b3d60 0x7f23d40af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:35.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.736+0000 7f23dbcb6640 10 monclient(hunting): picked mon.noname-a con 0x7f23d40b3d60 addr v2:172.21.15.19:3300/0 2023-10-06T14:26:35.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.736+0000 7f23dbcb6640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23d40b00c0 0x7f23d40b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:35.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.736+0000 7f23dbcb6640 10 monclient(hunting): picked mon.noname-b con 0x7f23d40b00c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:35.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.736+0000 7f23dbcb6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d40aebe0 con 0x7f23d40b3990 2023-10-06T14:26:35.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.736+0000 7f23dbcb6640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:35.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.736+0000 7f23dbcb6640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d40afd50 con 0x7f23d40b3d60 2023-10-06T14:26:35.741 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.736+0000 7f23dbcb6640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.736+0000 7f23dbcb6640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d4140dd0 con 0x7f23d40b00c0 2023-10-06T14:26:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.736+0000 7f23dbcb6640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.736+0000 7f23dbcb6640 10 monclient(hunting): authenticate will time out at 3323.922882s 2023-10-06T14:26:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23da253640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23d40b00c0 0x7f23d40b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23d9251640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f23d40b3d60 0x7f23d40af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23d9a52640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f23d40b3990 0x7f23d4141500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:34188/0 (socket says 172.21.15.19:34188) 2023-10-06T14:26:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23d9a52640 1 -- 172.21.15.19:0/2853442922 learned_addr learned my addr 172.21.15.19:0/2853442922 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T14:26:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23da253640 10 monclient(hunting): get_auth_request con 0x7f23d40b00c0 auth_method 0 2023-10-06T14:26:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23da253640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23da253640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23da253640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23d9251640 10 monclient(hunting): get_auth_request con 0x7f23d40b3d60 auth_method 0 2023-10-06T14:26:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23d9251640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23d9251640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23d9251640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23da253640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23da253640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.737+0000 7f23da253640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.738+0000 7f23d9251640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.738+0000 7f23d9251640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.738+0000 7f23d9251640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d8a50640 1 -- 172.21.15.19:0/2853442922 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124477251 0 0) 0x7f23d40aebe0 con 0x7f23d40b3990 2023-10-06T14:26:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d8a50640 10 monclient(hunting): _negotiate 2023-10-06T14:26:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d8a50640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d8a50640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d8a50640 10 monclient(hunting): my global_id is 20821 2023-10-06T14:26:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d8a50640 1 -- 172.21.15.19:0/2853442922 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23bc003520 con 0x7f23d40b3990 2023-10-06T14:26:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d9251640 10 monclient(hunting): handle_auth_done global_id 15705 payload 274 2023-10-06T14:26:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d9251640 1 -- 172.21.15.19:0/2853442922 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23d40b00c0 msgr2=0x7f23d40b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d9251640 1 --2- 172.21.15.19:0/2853442922 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23d40b00c0 0x7f23d40b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d9251640 1 -- 172.21.15.19:0/2853442922 >> v1:172.21.15.83:6789/0 conn(0x7f23d40b3990 legacy=0x7f23d4141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d9251640 10 monclient: _finish_hunting 0 2023-10-06T14:26:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d9251640 1 monclient: found mon.noname-a 2023-10-06T14:26:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d9251640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d9251640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T14:26:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23d9251640 1 -- 172.21.15.19:0/2853442922 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d4141070 con 0x7f23d40b3d60 2023-10-06T14:26:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.739+0000 7f23da253640 1 --2- 172.21.15.19:0/2853442922 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23d40b00c0 0x7f23d40b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T14:26:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.741+0000 7f23d9251640 1 --2- 172.21.15.19:0/2853442922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23d40b3d60 0x7f23d40af810 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 crypto rx=0x7f23c80097a0 tx=0x7f23c802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980483d5c0c70556 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23d8a50640 1 -- 172.21.15.19:0/2853442922 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c802e070 con 0x7f23d40b3d60 2023-10-06T14:26:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23d8a50640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23d8a50640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T14:26:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23d8a50640 10 monclient: dump: 2023-10-06T14:26:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:35.748 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23d8a50640 10 monclient: _finish_auth 0 2023-10-06T14:26:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23d8a50640 10 monclient: _check_auth_tickets 2023-10-06T14:26:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23d8a50640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23d8a50640 1 -- 172.21.15.19:0/2853442922 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c80315a0 con 0x7f23d40b3d60 2023-10-06T14:26:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23dbcb6640 5 monclient: authenticate success, global_id 15705 2023-10-06T14:26:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23dbcb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23d8a50640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23dbcb6640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23d8a50640 1 -- 172.21.15.19:0/2853442922 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8031930 con 0x7f23d40b3d60 2023-10-06T14:26:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23d8a50640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23dbcb6640 4 set_mon_vals no callback set 2023-10-06T14:26:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23d8a50640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T14:26:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.745+0000 7f23d8a50640 10 monclient: dump: 2023-10-06T14:26:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:35.750 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:35.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.746+0000 7f23dbcb6640 10 monclient: shutdown 2023-10-06T14:26:35.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.746+0000 7f23dbcb6640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:35.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.746+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2853442922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23d40b3d60 msgr2=0x7f23d40af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:35.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.746+0000 7f23dbcb6640 1 --2- 172.21.15.19:0/2853442922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23d40b3d60 0x7f23d40af810 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 crypto rx=0x7f23c80097a0 tx=0x7f23c802b840 comp rx=0 tx=0).stop 2023-10-06T14:26:35.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.746+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2853442922 shutdown_connections 2023-10-06T14:26:35.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.746+0000 7f23dbcb6640 1 --2- 172.21.15.19:0/2853442922 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23d40b00c0 0x7f23d40b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.753 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.746+0000 7f23dbcb6640 1 --2- 172.21.15.19:0/2853442922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23d40b3d60 0x7f23d40af810 unknown :-1 s=CLOSED pgs=3197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.746+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2853442922 >> 172.21.15.19:0/2853442922 conn(0x7f23d40b6e00 msgr2=0x7f23d40b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.746+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2853442922 shutdown_connections 2023-10-06T14:26:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.746+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2853442922 wait complete. 2023-10-06T14:26:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.746+0000 7f23dbcb6640 10 monclient: build_initial_monmap 2023-10-06T14:26:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.746+0000 7f23dbcb6640 10 monclient: monmap: 2023-10-06T14:26:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:35.755 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:35.755 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:35.755 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:35.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.747+0000 7f23dbcb6640 1 librados: starting msgr at 2023-10-06T14:26:35.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.747+0000 7f23dbcb6640 1 librados: starting objecter 2023-10-06T14:26:35.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.747+0000 7f23dbcb6640 1 Processor -- start 2023-10-06T14:26:35.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.747+0000 7f23dbcb6640 1 -- start start 2023-10-06T14:26:35.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.747+0000 7f23dbcb6640 1 librados: setting wanted keys 2023-10-06T14:26:35.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.747+0000 7f23dbcb6640 1 librados: calling monclient init 2023-10-06T14:26:35.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.747+0000 7f23dbcb6640 10 monclient: init 2023-10-06T14:26:35.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.747+0000 7f23dbcb6640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:35.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 10 monclient: _add_conns ranks=[2,0,1] 2023-10-06T14:26:35.755 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23d40b00c0 0x7f23d413d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 10 monclient(hunting): picked mon.noname-c con 0x7f23d40b00c0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23d40b3990 0x7f23d40a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 10 monclient(hunting): picked mon.noname-a con 0x7f23d40b3990 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d40b3d60 0x7f23d40aca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 10 monclient(hunting): picked mon.noname-b con 0x7f23d40b3d60 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40a9de0 con 0x7f23d40b3990 2023-10-06T14:26:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40abc20 con 0x7f23d40b3d60 2023-10-06T14:26:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40ac530 con 0x7f23d40b00c0 2023-10-06T14:26:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:35.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23dbcb6640 10 monclient(hunting): authenticate will time out at 3323.934416s 2023-10-06T14:26:35.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23d9251640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23d40b3990 0x7f23d40a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:35.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23d9a52640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23d40b00c0 0x7f23d413d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:35.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23da253640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d40b3d60 0x7f23d40aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:35.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23d9a52640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23d40b00c0 0x7f23d413d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:55510/0 (socket says 172.21.15.19:55510) 2023-10-06T14:26:35.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23d9a52640 1 -- 172.21.15.19:0/2429423615 learned_addr learned my addr 172.21.15.19:0/2429423615 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:35.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23d9a52640 10 monclient(hunting): get_auth_request con 0x7f23d40b00c0 auth_method 0 2023-10-06T14:26:35.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23d9a52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:35.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23d9a52640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:35.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23d9a52640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:35.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23d9251640 10 monclient(hunting): get_auth_request con 0x7f23d40b3990 auth_method 0 2023-10-06T14:26:35.757 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23d9251640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23d9251640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.748+0000 7f23d9251640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.749+0000 7f23da253640 10 monclient(hunting): get_auth_request con 0x7f23d40b3d60 auth_method 0 2023-10-06T14:26:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.749+0000 7f23da253640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.749+0000 7f23da253640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.749+0000 7f23da253640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.749+0000 7f23d9251640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.749+0000 7f23d9251640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.749+0000 7f23d9251640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.749+0000 7f23d9a52640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.749+0000 7f23d9a52640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.749+0000 7f23d9a52640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:35.759 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.749+0000 7f23da253640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:35.759 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.749+0000 7f23da253640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:35.759 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.749+0000 7f23da253640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:35.760 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.752+0000 7f23da253640 10 monclient(hunting): handle_auth_done global_id 20824 payload 824 2023-10-06T14:26:35.760 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.752+0000 7f23da253640 1 -- 172.21.15.19:0/2429423615 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23d40b00c0 msgr2=0x7f23d413d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:35.760 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.752+0000 7f23da253640 1 --2- 172.21.15.19:0/2429423615 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23d40b00c0 0x7f23d413d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.760 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.752+0000 7f23da253640 1 -- 172.21.15.19:0/2429423615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23d40b3990 msgr2=0x7f23d40a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:35.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.752+0000 7f23da253640 1 --2- 172.21.15.19:0/2429423615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23d40b3990 0x7f23d40a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:35.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.752+0000 7f23da253640 10 monclient: _finish_hunting 0 2023-10-06T14:26:35.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.752+0000 7f23da253640 1 monclient: found mon.noname-b 2023-10-06T14:26:35.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.752+0000 7f23da253640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:35.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.752+0000 7f23da253640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T14:26:35.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.752+0000 7f23da253640 1 -- 172.21.15.19:0/2429423615 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc009400 con 0x7f23d40b3d60 2023-10-06T14:26:35.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.754+0000 7f23da253640 1 --2- 172.21.15.19:0/2429423615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d40b3d60 0x7f23d40aca50 secure :-1 s=READY pgs=5130 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00e880 tx=0x7f23cc00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:35.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.754+0000 7f23d27fc640 10 client.?.objecter ms_handle_connect 0x7f23d40b3d60 2023-10-06T14:26:35.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.754+0000 7f23d27fc640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:35.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23d27fc640 1 -- 172.21.15.19:0/2429423615 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc004020 con 0x7f23d40b3d60 2023-10-06T14:26:35.761 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23d27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23d27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23d27fc640 10 monclient: dump: 2023-10-06T14:26:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:35.762 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23d27fc640 10 monclient: _finish_auth 0 2023-10-06T14:26:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23d27fc640 10 monclient: _check_auth_tickets 2023-10-06T14:26:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23d27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23d27fc640 1 -- 172.21.15.19:0/2429423615 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc0041a0 con 0x7f23d40b3d60 2023-10-06T14:26:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23d27fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23d27fc640 1 -- 172.21.15.19:0/2429423615 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc00c940 con 0x7f23d40b3d60 2023-10-06T14:26:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23dbcb6640 5 monclient: authenticate success, global_id 20824 2023-10-06T14:26:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23dbcb6640 10 monclient: _renew_subs 2023-10-06T14:26:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23dbcb6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2429423615 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c8009400 con 0x7f23d40b3d60 2023-10-06T14:26:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23d27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23d27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23d27fc640 10 monclient: dump: 2023-10-06T14:26:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:35.765 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:35.765 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:35.765 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:35.765 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23d3fff640 4 set_mon_vals no callback set 2023-10-06T14:26:35.765 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23dbcb6640 10 client.20824.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:35.765 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23dbcb6640 10 monclient: _renew_subs 2023-10-06T14:26:35.765 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23dbcb6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:35.765 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2429423615 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d413e170 con 0x7f23d40b3d60 2023-10-06T14:26:35.765 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.758+0000 7f23dbcb6640 1 librados: init done 2023-10-06T14:26:35.766 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.759+0000 7f23dbcb6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:35.766 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.759+0000 7f23dbcb6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:35.766 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.759+0000 7f23dbcb6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:35.766 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.759+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2429423615 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d40a86f0 con 0x7f23d40b3d60 2023-10-06T14:26:35.996 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.994+0000 7f23d27fc640 1 -- 172.21.15.19:0/2429423615 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f23cc00caa0 con 0x7f23d40b3d60 2023-10-06T14:26:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.995+0000 7f23d27fc640 1 --2- 172.21.15.19:0/2429423615 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f23c006c210 0x7f23c006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.995+0000 7f23d9a52640 1 --2- 172.21.15.19:0/2429423615 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f23c006c210 0x7f23c006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.995+0000 7f23d9a52640 10 monclient: get_auth_request con 0x7f23c006c210 auth_method 0 2023-10-06T14:26:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.996+0000 7f23d9a52640 1 --2- 172.21.15.19:0/2429423615 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f23c006c210 0x7f23c006e6c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f23c4005c00 tx=0x7f23c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.996+0000 7f23d27fc640 10 client.20824.objecter ms_handle_connect 0x7f23c006c210 2023-10-06T14:26:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.996+0000 7f23d27fc640 1 -- 172.21.15.19:0/2429423615 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(970..970 src has 714..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f23cc0585c0 con 0x7f23d40b3d60 2023-10-06T14:26:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.996+0000 7f23d27fc640 10 client.20824.objecter ms_dispatch 0x7f23d4000b90 osd_map(970..970 src has 714..970) v4 2023-10-06T14:26:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.996+0000 7f23d27fc640 3 client.20824.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:35.999 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.996+0000 7f23d27fc640 3 client.20824.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:35.999 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:35.997+0000 7f23d27fc640 20 client.20824.objecter dump_active .. 0 homeless 2023-10-06T14:26:36.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.006+0000 7f23d27fc640 1 -- 172.21.15.19:0/2429423615 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f23cc054900 con 0x7f23d40b3d60 2023-10-06T14:26:36.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.006+0000 7f23d27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:36.008 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.006+0000 7f23d27fc640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:26:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.191+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2429423615 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f23d40af110 con 0x7f23c006c210 2023-10-06T14:26:36.198 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.196+0000 7f23d27fc640 1 -- 172.21.15.19:0/2429423615 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+151409 (secure 0 0 0) 0x7f23c4001fe0 con 0x7f23c006c210 2023-10-06T14:26:36.199 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T14:26:36.200 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2023-10-06T14:26:36.201 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 10 librados: watch_flush enter 2023-10-06T14:26:36.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 10 librados: watch_flush exit 2023-10-06T14:26:36.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 20 client.20824.objecter shutdown clearing up homeless session... 2023-10-06T14:26:36.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 10 client.20824.objecter successfully canceled tick 2023-10-06T14:26:36.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2429423615 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f23c006c210 msgr2=0x7f23c006e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:36.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 1 --2- 172.21.15.19:0/2429423615 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f23c006c210 0x7f23c006e6c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f23c4005c00 tx=0x7f23c4009040 comp rx=0 tx=0).stop 2023-10-06T14:26:36.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 10 monclient: shutdown 2023-10-06T14:26:36.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:36.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2429423615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d40b3d60 msgr2=0x7f23d40aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:36.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 1 --2- 172.21.15.19:0/2429423615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d40b3d60 0x7f23d40aca50 secure :-1 s=READY pgs=5130 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00e880 tx=0x7f23cc00ed40 comp rx=0 tx=0).stop 2023-10-06T14:26:36.202 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2429423615 shutdown_connections 2023-10-06T14:26:36.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 1 --2- 172.21.15.19:0/2429423615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d40b3d60 0x7f23d40aca50 unknown :-1 s=CLOSED pgs=5130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:36.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 1 --2- 172.21.15.19:0/2429423615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23d40b3990 0x7f23d40a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:36.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 1 --2- 172.21.15.19:0/2429423615 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f23c006c210 0x7f23c006e6c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:36.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 1 --2- 172.21.15.19:0/2429423615 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f23d40b00c0 0x7f23d413d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:36.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2429423615 >> 172.21.15.19:0/2429423615 conn(0x7f23d40b6e00 msgr2=0x7f23d413a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:36.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2429423615 shutdown_connections 2023-10-06T14:26:36.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 1 -- 172.21.15.19:0/2429423615 wait complete. 2023-10-06T14:26:36.203 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:36.198+0000 7f23dbcb6640 1 librados: shutdown 2023-10-06T14:26:36.211 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":1978,"stamp":"2023-10-06T14:26:35.806241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":515000,"kb_used_data":45176,"kb_used_omap":0,"kb_used_meta":469760,"kb_avail":754459720,"statfs":{"total":773094113280,"available":772566753280,"internally_reserved":0,"allocated":46260224,"data_stored":24954866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":481034240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":90,"apply_latency_ms":90,"commit_latency_ns":90000000,"apply_latency_ns":90000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002439"},"pg_stats":[{"pgid":"8.14","version":"0'0","reported_seq":924,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:29.780674+0000","last_change":"2023-10-06T13:54:27.674098+0000","last_active":"2023-10-06T14:26:29.780674+0000","last_peered":"2023-10-06T14:26:29.780674+0000","last_clean":"2023-10-06T14:26:29.780674+0000","last_became_active":"2023-10-06T13:54:27.664679+0000","last_became_peered":"2023-10-06T13:54:27.664679+0000","last_unstale":"2023-10-06T14:26:29.780674+0000","last_undegraded":"2023-10-06T14:26:29.780674+0000","last_fullsized":"2023-10-06T14:26:29.780674+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":923,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.379601+0000","last_change":"2023-10-06T14:22:39.751625+0000","last_active":"2023-10-06T14:26:28.379601+0000","last_peered":"2023-10-06T14:26:28.379601+0000","last_clean":"2023-10-06T14:26:28.379601+0000","last_became_active":"2023-10-06T14:22:39.724017+0000","last_became_peered":"2023-10-06T14:22:39.724017+0000","last_unstale":"2023-10-06T14:26:28.379601+0000","last_undegraded":"2023-10-06T14:26:28.379601+0000","last_fullsized":"2023-10-06T14:26:28.379601+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":927,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.823322+0000","last_change":"2023-10-06T14:16:33.556535+0000","last_active":"2023-10-06T14:26:24.823322+0000","last_peered":"2023-10-06T14:26:24.823322+0000","last_clean":"2023-10-06T14:26:24.823322+0000","last_became_active":"2023-10-06T14:16:33.542923+0000","last_became_peered":"2023-10-06T14:16:33.542923+0000","last_unstale":"2023-10-06T14:26:24.823322+0000","last_undegraded":"2023-10-06T14:26:24.823322+0000","last_fullsized":"2023-10-06T14:26:24.823322+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:56:19.999321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":930,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.412974+0000","last_change":"2023-10-06T13:54:29.208698+0000","last_active":"2023-10-06T14:26:28.412974+0000","last_peered":"2023-10-06T14:26:28.412974+0000","last_clean":"2023-10-06T14:26:28.412974+0000","last_became_active":"2023-10-06T13:54:29.163161+0000","last_became_peered":"2023-10-06T13:54:29.163161+0000","last_unstale":"2023-10-06T14:26:28.412974+0000","last_undegraded":"2023-10-06T14:26:28.412974+0000","last_fullsized":"2023-10-06T14:26:28.412974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":922,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.353597+0000","last_change":"2023-10-06T14:22:39.220927+0000","last_active":"2023-10-06T14:26:28.353597+0000","last_peered":"2023-10-06T14:26:28.353597+0000","last_clean":"2023-10-06T14:26:28.353597+0000","last_became_active":"2023-10-06T14:22:39.128927+0000","last_became_peered":"2023-10-06T14:22:39.128927+0000","last_unstale":"2023-10-06T14:26:28.353597+0000","last_undegraded":"2023-10-06T14:26:28.353597+0000","last_fullsized":"2023-10-06T14:26:28.353597+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":923,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.361357+0000","last_change":"2023-10-06T14:22:39.468045+0000","last_active":"2023-10-06T14:26:28.361357+0000","last_peered":"2023-10-06T14:26:28.361357+0000","last_clean":"2023-10-06T14:26:28.361357+0000","last_became_active":"2023-10-06T14:22:39.418221+0000","last_became_peered":"2023-10-06T14:22:39.418221+0000","last_unstale":"2023-10-06T14:26:28.361357+0000","last_undegraded":"2023-10-06T14:26:28.361357+0000","last_fullsized":"2023-10-06T14:26:28.361357+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":923,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.364204+0000","last_change":"2023-10-06T14:22:39.210447+0000","last_active":"2023-10-06T14:26:28.364204+0000","last_peered":"2023-10-06T14:26:28.364204+0000","last_clean":"2023-10-06T14:26:28.364204+0000","last_became_active":"2023-10-06T14:22:39.195040+0000","last_became_peered":"2023-10-06T14:22:39.195040+0000","last_unstale":"2023-10-06T14:26:28.364204+0000","last_undegraded":"2023-10-06T14:26:28.364204+0000","last_fullsized":"2023-10-06T14:26:28.364204+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":928,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.813155+0000","last_change":"2023-10-06T14:16:33.305137+0000","last_active":"2023-10-06T14:26:24.813155+0000","last_peered":"2023-10-06T14:26:24.813155+0000","last_clean":"2023-10-06T14:26:24.813155+0000","last_became_active":"2023-10-06T14:16:33.300222+0000","last_became_peered":"2023-10-06T14:16:33.300222+0000","last_unstale":"2023-10-06T14:26:24.813155+0000","last_undegraded":"2023-10-06T14:26:24.813155+0000","last_fullsized":"2023-10-06T14:26:24.813155+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:34:52.750381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":922,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.377551+0000","last_change":"2023-10-06T14:22:39.745594+0000","last_active":"2023-10-06T14:26:28.377551+0000","last_peered":"2023-10-06T14:26:28.377551+0000","last_clean":"2023-10-06T14:26:28.377551+0000","last_became_active":"2023-10-06T14:22:39.712413+0000","last_became_peered":"2023-10-06T14:22:39.712413+0000","last_unstale":"2023-10-06T14:26:28.377551+0000","last_undegraded":"2023-10-06T14:26:28.377551+0000","last_fullsized":"2023-10-06T14:26:28.377551+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":909,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.544613+0000","last_change":"2023-10-06T13:54:19.475064+0000","last_active":"2023-10-06T14:26:24.544613+0000","last_peered":"2023-10-06T14:26:24.544613+0000","last_clean":"2023-10-06T14:26:24.544613+0000","last_became_active":"2023-10-06T13:54:19.427005+0000","last_became_peered":"2023-10-06T13:54:19.427005+0000","last_unstale":"2023-10-06T14:26:24.544613+0000","last_undegraded":"2023-10-06T14:26:24.544613+0000","last_fullsized":"2023-10-06T14:26:24.544613+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:21.985890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":911,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.581995+0000","last_change":"2023-10-06T13:54:19.536231+0000","last_active":"2023-10-06T14:26:24.581995+0000","last_peered":"2023-10-06T14:26:24.581995+0000","last_clean":"2023-10-06T14:26:24.581995+0000","last_became_active":"2023-10-06T13:54:19.531077+0000","last_became_peered":"2023-10-06T13:54:19.531077+0000","last_unstale":"2023-10-06T14:26:24.581995+0000","last_undegraded":"2023-10-06T14:26:24.581995+0000","last_fullsized":"2023-10-06T14:26:24.581995+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:05.864633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":922,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.817123+0000","last_change":"2023-10-06T14:06:28.635986+0000","last_active":"2023-10-06T14:26:24.817123+0000","last_peered":"2023-10-06T14:26:24.817123+0000","last_clean":"2023-10-06T14:26:24.817123+0000","last_became_active":"2023-10-06T14:06:28.630618+0000","last_became_peered":"2023-10-06T14:06:28.630618+0000","last_unstale":"2023-10-06T14:26:24.817123+0000","last_undegraded":"2023-10-06T14:26:24.817123+0000","last_fullsized":"2023-10-06T14:26:24.817123+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:32.474746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":931,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.606016+0000","last_change":"2023-10-06T13:54:21.830131+0000","last_active":"2023-10-06T14:26:24.606016+0000","last_peered":"2023-10-06T14:26:24.606016+0000","last_clean":"2023-10-06T14:26:24.606016+0000","last_became_active":"2023-10-06T13:54:21.813915+0000","last_became_peered":"2023-10-06T13:54:21.813915+0000","last_unstale":"2023-10-06T14:26:24.606016+0000","last_undegraded":"2023-10-06T14:26:24.606016+0000","last_fullsized":"2023-10-06T14:26:24.606016+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:21:55.469659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":925,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.383528+0000","last_change":"2023-10-06T14:22:39.679043+0000","last_active":"2023-10-06T14:26:28.383528+0000","last_peered":"2023-10-06T14:26:28.383528+0000","last_clean":"2023-10-06T14:26:28.383528+0000","last_became_active":"2023-10-06T14:22:39.673285+0000","last_became_peered":"2023-10-06T14:22:39.673285+0000","last_unstale":"2023-10-06T14:26:28.383528+0000","last_undegraded":"2023-10-06T14:26:28.383528+0000","last_fullsized":"2023-10-06T14:26:28.383528+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":916,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.818620+0000","last_change":"2023-10-06T14:16:33.403287+0000","last_active":"2023-10-06T14:26:24.818620+0000","last_peered":"2023-10-06T14:26:24.818620+0000","last_clean":"2023-10-06T14:26:24.818620+0000","last_became_active":"2023-10-06T14:16:33.392050+0000","last_became_peered":"2023-10-06T14:16:33.392050+0000","last_unstale":"2023-10-06T14:26:24.818620+0000","last_undegraded":"2023-10-06T14:26:24.818620+0000","last_fullsized":"2023-10-06T14:26:24.818620+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:29:10.779601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":925,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.813387+0000","last_change":"2023-10-06T14:05:13.253436+0000","last_active":"2023-10-06T14:26:24.813387+0000","last_peered":"2023-10-06T14:26:24.813387+0000","last_clean":"2023-10-06T14:26:24.813387+0000","last_became_active":"2023-10-06T14:05:13.248166+0000","last_became_peered":"2023-10-06T14:05:13.248166+0000","last_unstale":"2023-10-06T14:26:24.813387+0000","last_undegraded":"2023-10-06T14:26:24.813387+0000","last_fullsized":"2023-10-06T14:26:24.813387+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:51:29.023023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":909,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.584532+0000","last_change":"2023-10-06T13:54:26.224788+0000","last_active":"2023-10-06T14:26:24.584532+0000","last_peered":"2023-10-06T14:26:24.584532+0000","last_clean":"2023-10-06T14:26:24.584532+0000","last_became_active":"2023-10-06T13:54:26.218851+0000","last_became_peered":"2023-10-06T13:54:26.218851+0000","last_unstale":"2023-10-06T14:26:24.584532+0000","last_undegraded":"2023-10-06T14:26:24.584532+0000","last_fullsized":"2023-10-06T14:26:24.584532+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:43:20.812836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":942,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.381372+0000","last_change":"2023-10-06T14:22:39.234795+0000","last_active":"2023-10-06T14:26:28.381372+0000","last_peered":"2023-10-06T14:26:28.381372+0000","last_clean":"2023-10-06T14:26:28.381372+0000","last_became_active":"2023-10-06T14:22:39.109893+0000","last_became_peered":"2023-10-06T14:22:39.109893+0000","last_unstale":"2023-10-06T14:26:28.381372+0000","last_undegraded":"2023-10-06T14:26:28.381372+0000","last_fullsized":"2023-10-06T14:26:28.381372+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":928,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.824871+0000","last_change":"2023-10-06T14:16:33.559587+0000","last_active":"2023-10-06T14:26:24.824871+0000","last_peered":"2023-10-06T14:26:24.824871+0000","last_clean":"2023-10-06T14:26:24.824871+0000","last_became_active":"2023-10-06T14:16:33.551586+0000","last_became_peered":"2023-10-06T14:16:33.551586+0000","last_unstale":"2023-10-06T14:26:24.824871+0000","last_undegraded":"2023-10-06T14:26:24.824871+0000","last_fullsized":"2023-10-06T14:26:24.824871+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:53.251968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":921,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.820207+0000","last_change":"2023-10-06T14:16:33.406336+0000","last_active":"2023-10-06T14:26:24.820207+0000","last_peered":"2023-10-06T14:26:24.820207+0000","last_clean":"2023-10-06T14:26:24.820207+0000","last_became_active":"2023-10-06T14:16:33.398349+0000","last_became_peered":"2023-10-06T14:16:33.398349+0000","last_unstale":"2023-10-06T14:26:24.820207+0000","last_undegraded":"2023-10-06T14:26:24.820207+0000","last_fullsized":"2023-10-06T14:26:24.820207+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:24:58.230306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":931,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.402231+0000","last_change":"2023-10-06T13:54:31.926742+0000","last_active":"2023-10-06T14:26:28.402231+0000","last_peered":"2023-10-06T14:26:28.402231+0000","last_clean":"2023-10-06T14:26:28.402231+0000","last_became_active":"2023-10-06T13:54:31.908571+0000","last_became_peered":"2023-10-06T13:54:31.908571+0000","last_unstale":"2023-10-06T14:26:28.402231+0000","last_undegraded":"2023-10-06T14:26:28.402231+0000","last_fullsized":"2023-10-06T14:26:28.402231+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":931,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.367537+0000","last_change":"2023-10-06T14:22:39.277750+0000","last_active":"2023-10-06T14:26:28.367537+0000","last_peered":"2023-10-06T14:26:28.367537+0000","last_clean":"2023-10-06T14:26:28.367537+0000","last_became_active":"2023-10-06T14:22:39.078831+0000","last_became_peered":"2023-10-06T14:22:39.078831+0000","last_unstale":"2023-10-06T14:26:28.367537+0000","last_undegraded":"2023-10-06T14:26:28.367537+0000","last_fullsized":"2023-10-06T14:26:28.367537+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":921,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.804141+0000","last_change":"2023-10-06T14:05:13.235606+0000","last_active":"2023-10-06T14:26:24.804141+0000","last_peered":"2023-10-06T14:26:24.804141+0000","last_clean":"2023-10-06T14:26:24.804141+0000","last_became_active":"2023-10-06T14:05:13.230098+0000","last_became_peered":"2023-10-06T14:05:13.230098+0000","last_unstale":"2023-10-06T14:26:24.804141+0000","last_undegraded":"2023-10-06T14:26:24.804141+0000","last_fullsized":"2023-10-06T14:26:24.804141+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:28:31.276617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":930,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:30.007020+0000","last_change":"2023-10-06T14:16:33.641985+0000","last_active":"2023-10-06T14:26:30.007020+0000","last_peered":"2023-10-06T14:26:30.007020+0000","last_clean":"2023-10-06T14:26:30.007020+0000","last_became_active":"2023-10-06T14:16:33.635681+0000","last_became_peered":"2023-10-06T14:16:33.635681+0000","last_unstale":"2023-10-06T14:26:30.007020+0000","last_undegraded":"2023-10-06T14:26:30.007020+0000","last_fullsized":"2023-10-06T14:26:30.007020+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:42.455510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":930,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.814755+0000","last_change":"2023-10-06T14:16:33.275781+0000","last_active":"2023-10-06T14:26:24.814755+0000","last_peered":"2023-10-06T14:26:24.814755+0000","last_clean":"2023-10-06T14:26:24.814755+0000","last_became_active":"2023-10-06T14:16:33.269648+0000","last_became_peered":"2023-10-06T14:16:33.269648+0000","last_unstale":"2023-10-06T14:26:24.814755+0000","last_undegraded":"2023-10-06T14:26:24.814755+0000","last_fullsized":"2023-10-06T14:26:24.814755+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:25.574583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":941,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.355295+0000","last_change":"2023-10-06T14:22:39.170180+0000","last_active":"2023-10-06T14:26:28.355295+0000","last_peered":"2023-10-06T14:26:28.355295+0000","last_clean":"2023-10-06T14:26:28.355295+0000","last_became_active":"2023-10-06T14:22:39.121176+0000","last_became_peered":"2023-10-06T14:22:39.121176+0000","last_unstale":"2023-10-06T14:26:28.355295+0000","last_undegraded":"2023-10-06T14:26:28.355295+0000","last_fullsized":"2023-10-06T14:26:28.355295+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":936,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:34.523930+0000","last_change":"2023-10-06T14:26:34.528621+0000","last_active":"2023-10-06T14:26:34.523930+0000","last_peered":"2023-10-06T14:26:34.523930+0000","last_clean":"2023-10-06T14:26:34.523930+0000","last_became_active":"2023-10-06T13:54:27.642873+0000","last_became_peered":"2023-10-06T13:54:27.642873+0000","last_unstale":"2023-10-06T14:26:34.523930+0000","last_undegraded":"2023-10-06T14:26:34.523930+0000","last_fullsized":"2023-10-06T14:26:34.523930+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:34.522838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:34.522838+0000","last_clean_scrub_stamp":"2023-10-06T14:26:34.522838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:59:08.227916+0000","scrub_duration":0.081521365999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":929,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.371814+0000","last_change":"2023-10-06T14:22:39.756236+0000","last_active":"2023-10-06T14:26:28.371814+0000","last_peered":"2023-10-06T14:26:28.371814+0000","last_clean":"2023-10-06T14:26:28.371814+0000","last_became_active":"2023-10-06T14:22:39.733473+0000","last_became_peered":"2023-10-06T14:22:39.733473+0000","last_unstale":"2023-10-06T14:26:28.371814+0000","last_undegraded":"2023-10-06T14:26:28.371814+0000","last_fullsized":"2023-10-06T14:26:28.371814+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"970'78","reported_seq":1070,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:34.515865+0000","last_change":"2023-10-06T14:26:34.521747+0000","last_active":"2023-10-06T14:26:34.515865+0000","last_peered":"2023-10-06T14:26:34.515865+0000","last_clean":"2023-10-06T14:26:34.515865+0000","last_became_active":"2023-10-06T14:16:33.632776+0000","last_became_peered":"2023-10-06T14:16:33.632776+0000","last_unstale":"2023-10-06T14:26:34.515865+0000","last_undegraded":"2023-10-06T14:26:34.515865+0000","last_fullsized":"2023-10-06T14:26:34.515865+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"970'78","last_scrub_stamp":"2023-10-06T14:26:34.514526+0000","last_deep_scrub":"970'78","last_deep_scrub_stamp":"2023-10-06T14:26:34.514526+0000","last_clean_scrub_stamp":"2023-10-06T14:26:34.514526+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:00.523848+0000","scrub_duration":0.114787636,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":909,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.575703+0000","last_change":"2023-10-06T13:54:26.199772+0000","last_active":"2023-10-06T14:26:24.575703+0000","last_peered":"2023-10-06T14:26:24.575703+0000","last_clean":"2023-10-06T14:26:24.575703+0000","last_became_active":"2023-10-06T13:54:26.193354+0000","last_became_peered":"2023-10-06T13:54:26.193354+0000","last_unstale":"2023-10-06T14:26:24.575703+0000","last_undegraded":"2023-10-06T14:26:24.575703+0000","last_fullsized":"2023-10-06T14:26:24.575703+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:11.930838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":932,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.586726+0000","last_change":"2023-10-06T13:54:22.015064+0000","last_active":"2023-10-06T14:26:24.586726+0000","last_peered":"2023-10-06T14:26:24.586726+0000","last_clean":"2023-10-06T14:26:24.586726+0000","last_became_active":"2023-10-06T13:54:22.005243+0000","last_became_peered":"2023-10-06T13:54:22.005243+0000","last_unstale":"2023-10-06T14:26:24.586726+0000","last_undegraded":"2023-10-06T14:26:24.586726+0000","last_fullsized":"2023-10-06T14:26:24.586726+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:46:20.602620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"43'7","reported_seq":939,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.531781+0000","last_change":"2023-10-06T14:05:13.464997+0000","last_active":"2023-10-06T14:26:24.531781+0000","last_peered":"2023-10-06T14:26:24.531781+0000","last_clean":"2023-10-06T14:26:24.531781+0000","last_became_active":"2023-10-06T14:05:13.458219+0000","last_became_peered":"2023-10-06T14:05:13.458219+0000","last_unstale":"2023-10-06T14:26:24.531781+0000","last_undegraded":"2023-10-06T14:26:24.531781+0000","last_fullsized":"2023-10-06T14:26:24.531781+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:40:26.823980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":932,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:29.892468+0000","last_change":"2023-10-06T14:22:39.278351+0000","last_active":"2023-10-06T14:26:29.892468+0000","last_peered":"2023-10-06T14:26:29.892468+0000","last_clean":"2023-10-06T14:26:29.892468+0000","last_became_active":"2023-10-06T14:22:39.269435+0000","last_became_peered":"2023-10-06T14:22:39.269435+0000","last_unstale":"2023-10-06T14:26:29.892468+0000","last_undegraded":"2023-10-06T14:26:29.892468+0000","last_fullsized":"2023-10-06T14:26:29.892468+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:10.817190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":922,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.586085+0000","last_change":"2023-10-06T14:16:33.659581+0000","last_active":"2023-10-06T14:26:24.586085+0000","last_peered":"2023-10-06T14:26:24.586085+0000","last_clean":"2023-10-06T14:26:24.586085+0000","last_became_active":"2023-10-06T14:16:33.650127+0000","last_became_peered":"2023-10-06T14:16:33.650127+0000","last_unstale":"2023-10-06T14:26:24.586085+0000","last_undegraded":"2023-10-06T14:26:24.586085+0000","last_fullsized":"2023-10-06T14:26:24.586085+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:45.333800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":911,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.547444+0000","last_change":"2023-10-06T13:54:22.165757+0000","last_active":"2023-10-06T14:26:24.547444+0000","last_peered":"2023-10-06T14:26:24.547444+0000","last_clean":"2023-10-06T14:26:24.547444+0000","last_became_active":"2023-10-06T13:54:22.160421+0000","last_became_peered":"2023-10-06T13:54:22.160421+0000","last_unstale":"2023-10-06T14:26:24.547444+0000","last_undegraded":"2023-10-06T14:26:24.547444+0000","last_fullsized":"2023-10-06T14:26:24.547444+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:51.552552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":932,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.406384+0000","last_change":"2023-10-06T13:54:25.399248+0000","last_active":"2023-10-06T14:26:28.406384+0000","last_peered":"2023-10-06T14:26:28.406384+0000","last_clean":"2023-10-06T14:26:28.406384+0000","last_became_active":"2023-10-06T13:54:25.349894+0000","last_became_peered":"2023-10-06T13:54:25.349894+0000","last_unstale":"2023-10-06T14:26:28.406384+0000","last_undegraded":"2023-10-06T14:26:28.406384+0000","last_fullsized":"2023-10-06T14:26:28.406384+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":945,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.596774+0000","last_change":"2023-10-06T14:22:39.984989+0000","last_active":"2023-10-06T14:26:24.596774+0000","last_peered":"2023-10-06T14:26:24.596774+0000","last_clean":"2023-10-06T14:26:24.596774+0000","last_became_active":"2023-10-06T14:22:39.979167+0000","last_became_peered":"2023-10-06T14:22:39.979167+0000","last_unstale":"2023-10-06T14:26:24.596774+0000","last_undegraded":"2023-10-06T14:26:24.596774+0000","last_fullsized":"2023-10-06T14:26:24.596774+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":836,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:55:13.743000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":941,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.582710+0000","last_change":"2023-10-06T14:05:13.200912+0000","last_active":"2023-10-06T14:26:24.582710+0000","last_peered":"2023-10-06T14:26:24.582710+0000","last_clean":"2023-10-06T14:26:24.582710+0000","last_became_active":"2023-10-06T14:05:13.191757+0000","last_became_peered":"2023-10-06T14:05:13.191757+0000","last_unstale":"2023-10-06T14:26:24.582710+0000","last_undegraded":"2023-10-06T14:26:24.582710+0000","last_fullsized":"2023-10-06T14:26:24.582710+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:21:31.319110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":910,"reported_epoch":969,"state":"active+clean","last_fresh":"2023-10-06T14:26:24.538144+0000","last_change":"2023-10-06T13:54:22.153461+0000","last_active":"2023-10-06T14:26:24.538144+0000","last_peered":"2023-10-06T14:26:24.538144+0000","last_clean":"2023-10-06T14:26:24.538144+0000","last_became_active":"2023-10-06T13:54:22.144576+0000","last_became_peered":"2023-10-06T13:54:22.144576+0000","last_unstale":"2023-10-06T14:26:24.538144+0000","last_undegraded":"2023-10-06T14:26:24.538144+0000","last_fullsized":"2023-10-06T14:26:24.538144+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:55:12.616531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":930,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.408991+0000","last_change":"2023-10-06T13:54:19.740406+0000","last_active":"2023-10-06T14:26:28.408991+0000","last_peered":"2023-10-06T14:26:28.408991+0000","last_clean":"2023-10-06T14:26:28.408991+0000","last_became_active":"2023-10-06T13:54:19.734381+0000","last_became_peered":"2023-10-06T13:54:19.734381+0000","last_unstale":"2023-10-06T14:26:28.408991+0000","last_undegraded":"2023-10-06T14:26:28.408991+0000","last_fullsized":"2023-10-06T14:26:28.408991+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":948,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:30.418675+0000","last_change":"2023-10-06T14:26:30.426357+0000","last_active":"2023-10-06T14:26:30.418675+0000","last_peered":"2023-10-06T14:26:30.418675+0000","last_clean":"2023-10-06T14:26:30.418675+0000","last_became_active":"2023-10-06T14:05:13.442612+0000","last_became_peered":"2023-10-06T14:05:13.442612+0000","last_unstale":"2023-10-06T14:26:30.418675+0000","last_undegraded":"2023-10-06T14:26:30.418675+0000","last_fullsized":"2023-10-06T14:26:30.418675+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:30.417803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:30.417803+0000","last_clean_scrub_stamp":"2023-10-06T14:26:30.417803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:23:39.298614+0000","scrub_duration":0.036831065000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719477124,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59132,"kb_used_data":5940,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312708,"statfs":{"total":96636764160,"available":96576212992,"internally_reserved":0,"allocated":6082560,"data_stored":3417737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":11.946,"5min":12.488,"15min":13.279},"min":{"1min":6.3920000000000003,"5min":4.6070000000000002,"15min":4.6070000000000002},"max":{"1min":52.079999999999998,"5min":91.744,"15min":109.691},"last":4.6079999999999997},{"interface":"front","average":{"1min":12.875999999999999,"5min":12.664,"15min":14.576000000000001},"min":{"1min":4.1870000000000003,"5min":4.1870000000000003,"15min":4.1870000000000003},"max":{"1min":44.232999999999997,"5min":67.936000000000007,"15min":166.33699999999999},"last":11.148999999999999}]},{"osd":2,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":12.112,"5min":11.595000000000001,"15min":13.598000000000001},"min":{"1min":7.9850000000000003,"5min":6.3920000000000003,"15min":4.9800000000000004},"max":{"1min":18.631,"5min":64.661000000000001,"15min":155.68799999999999},"last":8.3960000000000008},{"interface":"front","average":{"1min":15.757999999999999,"5min":13.82,"15min":16.335000000000001},"min":{"1min":9.0660000000000007,"5min":7.3049999999999997,"15min":6.6059999999999999},"max":{"1min":47.368000000000002,"5min":92.335999999999999,"15min":212.16399999999999},"last":17.425000000000001}]},{"osd":3,"last update":"Fri Oct 6 14:25:44 2023","interfaces":[{"interface":"back","average":{"1min":23.273,"5min":29.661000000000001,"15min":33.582999999999998},"min":{"1min":9.6639999999999997,"5min":8.6850000000000005,"15min":8.6850000000000005},"max":{"1min":126.22799999999999,"5min":310.19,"15min":310.19},"last":11.871},{"interface":"front","average":{"1min":18.346,"5min":18.327999999999999,"15min":18.873999999999999},"min":{"1min":8.5370000000000008,"5min":7.8399999999999999,"15min":7.8399999999999999},"max":{"1min":65.375,"5min":65.375,"15min":65.375},"last":14.141}]},{"osd":4,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":15290.522000000001,"5min":3070.721,"15min":1038.3879999999999},"min":{"1min":9.2720000000000002,"5min":7.8520000000000003,"15min":7.0190000000000001},"max":{"1min":39693.332999999999,"5min":39693.332999999999,"15min":39693.332999999999},"last":12.975},{"interface":"front","average":{"1min":15323.151,"5min":3078.1880000000001,"15min":1039.8889999999999},"min":{"1min":14.579000000000001,"5min":8.5679999999999996,"15min":7.3239999999999998},"max":{"1min":39697.402000000002,"5min":39697.402000000002,"15min":39697.402000000002},"last":9.3650000000000002}]},{"osd":5,"last update":"Fri Oct 6 14:25:44 2023","interfaces":[{"interface":"back","average":{"1min":21.614999999999998,"5min":17.170000000000002,"15min":92.123000000000005},"min":{"1min":9.1199999999999992,"5min":9.1199999999999992,"15min":9.1199999999999992},"max":{"1min":69.561999999999998,"5min":69.561999999999998,"15min":8890.4069999999992},"last":14.736000000000001},{"interface":"front","average":{"1min":25.356000000000002,"5min":17.224,"15min":92.638999999999996},"min":{"1min":8.3140000000000001,"5min":7.8620000000000001,"15min":7.8620000000000001},"max":{"1min":139.78800000000001,"5min":139.78800000000001,"15min":8888.7939999999999},"last":8.6720000000000006}]},{"osd":6,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.563000000000001,"5min":15.667999999999999,"15min":1233.97},"min":{"1min":9.9359999999999999,"5min":8.4710000000000001,"15min":7.3529999999999998},"max":{"1min":20.318000000000001,"5min":78.221999999999994,"15min":47304.226999999999},"last":12.425000000000001},{"interface":"front","average":{"1min":17.091000000000001,"5min":18.989000000000001,"15min":1237.328},"min":{"1min":11.202999999999999,"5min":10.666,"15min":8.9949999999999992},"max":{"1min":24.579000000000001,"5min":116.84999999999999,"15min":47302.955999999998},"last":13.553000000000001}]},{"osd":7,"last update":"Fri Oct 6 14:26:33 2023","interfaces":[{"interface":"back","average":{"1min":17.274999999999999,"5min":20.236000000000001,"15min":22.026},"min":{"1min":11.840999999999999,"5min":10.992000000000001,"15min":9.0250000000000004},"max":{"1min":23.812000000000001,"5min":78.769000000000005,"15min":172.16300000000001},"last":16.5},{"interface":"front","average":{"1min":15.042999999999999,"5min":20.027999999999999,"15min":21.492000000000001},"min":{"1min":9.7680000000000007,"5min":9.7680000000000007,"15min":9.7560000000000002},"max":{"1min":30.442,"5min":134.17400000000001,"15min":180.739},"last":10.585000000000001}]}]},{"osd":0,"up_from":16,"seq":68719477140,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58740,"kb_used_data":5548,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313100,"statfs":{"total":96636764160,"available":96576614400,"internally_reserved":0,"allocated":5681152,"data_stored":3019897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 14:26:15 2023","interfaces":[{"interface":"back","average":{"1min":19.928999999999998,"5min":17.440999999999999,"15min":16.654},"min":{"1min":6.7800000000000002,"5min":5.4980000000000002,"15min":4.9720000000000004},"max":{"1min":97.855999999999995,"5min":135.13800000000001,"15min":342.61399999999998},"last":10.289},{"interface":"front","average":{"1min":21.222000000000001,"5min":19.111999999999998,"15min":17.628},"min":{"1min":5.9409999999999998,"5min":5.8810000000000002,"15min":5.5960000000000001},"max":{"1min":144.221,"5min":144.221,"15min":337.48000000000002},"last":14.057}]},{"osd":2,"last update":"Fri Oct 6 14:26:25 2023","interfaces":[{"interface":"back","average":{"1min":24.416,"5min":21.916,"15min":19.238},"min":{"1min":5.5030000000000001,"5min":5.5030000000000001,"15min":5.5030000000000001},"max":{"1min":231.68000000000001,"5min":231.68000000000001,"15min":231.68000000000001},"last":15.834},{"interface":"front","average":{"1min":20.329000000000001,"5min":17.545000000000002,"15min":17.381},"min":{"1min":6.6470000000000002,"5min":5.718,"15min":5.718},"max":{"1min":121.98399999999999,"5min":181.06700000000001,"15min":181.06700000000001},"last":8.0199999999999996}]},{"osd":3,"last update":"Fri Oct 6 14:25:42 2023","interfaces":[{"interface":"back","average":{"1min":30.332000000000001,"5min":24.734999999999999,"15min":25.606999999999999},"min":{"1min":7.5949999999999998,"5min":7.4139999999999997,"15min":7.4139999999999997},"max":{"1min":189.10499999999999,"5min":189.10499999999999,"15min":189.10499999999999},"last":17.105},{"interface":"front","average":{"1min":31.155000000000001,"5min":24.603999999999999,"15min":24.998000000000001},"min":{"1min":10.960000000000001,"5min":6.9850000000000003,"15min":6.9850000000000003},"max":{"1min":163.60900000000001,"5min":163.60900000000001,"15min":163.60900000000001},"last":18.524999999999999}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":3655.643,"5min":746.56799999999998,"15min":262.995},"min":{"1min":13.273999999999999,"5min":8.7420000000000009,"15min":6.9269999999999996},"max":{"1min":39913.453999999998,"5min":39913.453999999998,"15min":39913.453999999998},"last":12.682},{"interface":"front","average":{"1min":3644.9029999999998,"5min":742.30499999999995,"15min":261.23700000000002},"min":{"1min":7.2050000000000001,"5min":7.2050000000000001,"15min":7.2050000000000001},"max":{"1min":39912.421999999999,"5min":39912.421999999999,"15min":39912.421999999999},"last":9.6039999999999992}]},{"osd":5,"last update":"Fri Oct 6 14:25:49 2023","interfaces":[{"interface":"back","average":{"1min":15.067,"5min":16.125,"15min":71.585999999999999},"min":{"1min":8.2599999999999998,"5min":7.5209999999999999,"15min":7.5209999999999999},"max":{"1min":56.817,"5min":142.267,"15min":6380.991},"last":20.274999999999999},{"interface":"front","average":{"1min":24.161000000000001,"5min":20.405000000000001,"15min":66.221000000000004},"min":{"1min":10.645,"5min":9.9390000000000001,"15min":9.4550000000000001},"max":{"1min":133.797,"5min":145.59299999999999,"15min":6266.0159999999996},"last":12.039999999999999}]},{"osd":6,"last update":"Fri Oct 6 14:26:05 2023","interfaces":[{"interface":"back","average":{"1min":13.15,"5min":20.032,"15min":725.85500000000002},"min":{"1min":7.7789999999999999,"5min":7.1479999999999997,"15min":7.1479999999999997},"max":{"1min":26.087,"5min":177.73699999999999,"15min":46416.196000000004},"last":11.42},{"interface":"front","average":{"1min":15.784000000000001,"5min":22.670000000000002,"15min":724.05899999999997},"min":{"1min":10.669,"5min":8.6609999999999996,"15min":6.9459999999999997},"max":{"1min":21.206,"5min":181.625,"15min":46403.283000000003},"last":19.146000000000001}]},{"osd":7,"last update":"Fri Oct 6 14:26:33 2023","interfaces":[{"interface":"back","average":{"1min":23.925000000000001,"5min":24.867999999999999,"15min":1322.9580000000001},"min":{"1min":10.121,"5min":10.121,"15min":9.8209999999999997},"max":{"1min":96.147000000000006,"5min":178.86500000000001,"15min":67028.107999999993},"last":16.408999999999999},{"interface":"front","average":{"1min":19.460000000000001,"5min":20.870999999999999,"15min":1320.289},"min":{"1min":7.8730000000000002,"5min":7.8730000000000002,"15min":7.8730000000000002},"max":{"1min":89.558999999999997,"5min":178.38900000000001,"15min":67001.737999999998},"last":14.643000000000001}]}]},{"osd":4,"up_from":430,"seq":1846835937656,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58744,"kb_used_data":5552,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313096,"statfs":{"total":96636764160,"available":96576610304,"internally_reserved":0,"allocated":5685248,"data_stored":3019902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":61.039999999999999,"5min":21.318999999999999,"15min":15.34},"min":{"1min":7.0979999999999999,"5min":4.508,"15min":4.085},"max":{"1min":469.096,"5min":469.096,"15min":469.096},"last":5.343},{"interface":"front","average":{"1min":60.287999999999997,"5min":20.288,"15min":17.068999999999999},"min":{"1min":8.5890000000000004,"5min":4.758,"15min":4.5999999999999996},"max":{"1min":423.42000000000002,"5min":423.42000000000002,"15min":423.42000000000002},"last":10.332000000000001}]},{"osd":1,"last update":"Fri Oct 6 14:26:13 2023","interfaces":[{"interface":"back","average":{"1min":81.888000000000005,"5min":27.257000000000001,"15min":19.550000000000001},"min":{"1min":7.3289999999999997,"5min":5.9850000000000003,"15min":4.9390000000000001},"max":{"1min":469.72800000000001,"5min":469.72800000000001,"15min":469.72800000000001},"last":8.673},{"interface":"front","average":{"1min":77.108999999999995,"5min":26.469999999999999,"15min":21.765999999999998},"min":{"1min":5.9320000000000004,"5min":5.9320000000000004,"15min":5.9320000000000004},"max":{"1min":425.584,"5min":425.584,"15min":425.584},"last":14.696999999999999}]},{"osd":2,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":113.371,"5min":34.213000000000001,"15min":23.704999999999998},"min":{"1min":7.6429999999999998,"5min":6.8159999999999998,"15min":6.4770000000000003},"max":{"1min":530.30399999999997,"5min":530.30399999999997,"15min":530.30399999999997},"last":9.7490000000000006},{"interface":"front","average":{"1min":117.30200000000001,"5min":37.220999999999997,"15min":26.550999999999998},"min":{"1min":10.667999999999999,"5min":7.0750000000000002,"15min":6.5780000000000003},"max":{"1min":530.86099999999999,"5min":530.86099999999999,"15min":530.86099999999999},"last":16.358000000000001}]},{"osd":3,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":58.695,"5min":25.254999999999999,"15min":19.239000000000001},"min":{"1min":11.391,"5min":9.9220000000000006,"15min":9.9220000000000006},"max":{"1min":470.80099999999999,"5min":470.80099999999999,"15min":470.80099999999999},"last":11.565},{"interface":"front","average":{"1min":49.429000000000002,"5min":22.123999999999999,"15min":17.120000000000001},"min":{"1min":6.7489999999999997,"5min":6.7489999999999997,"15min":6.7489999999999997},"max":{"1min":422.88,"5min":422.88,"15min":422.88},"last":14.066000000000001}]},{"osd":5,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":58.246000000000002,"5min":23.637,"15min":76.748000000000005},"min":{"1min":8.8409999999999993,"5min":8.2230000000000008,"15min":5.9509999999999996},"max":{"1min":431.57900000000001,"5min":431.57900000000001,"15min":8813.2510000000002},"last":17.484000000000002},{"interface":"front","average":{"1min":62.423000000000002,"5min":22.936,"15min":77.950000000000003},"min":{"1min":6.5590000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"max":{"1min":537.68399999999997,"5min":537.68399999999997,"15min":8811.7579999999998},"last":13.31}]},{"osd":6,"last update":"Fri Oct 6 14:26:13 2023","interfaces":[{"interface":"back","average":{"1min":96.150999999999996,"5min":31.527000000000001,"15min":920.85000000000002},"min":{"1min":9.7840000000000007,"5min":6.9470000000000001,"15min":6.9470000000000001},"max":{"1min":563.85900000000004,"5min":563.85900000000004,"15min":44817.595000000001},"last":18.760000000000002},{"interface":"front","average":{"1min":84.043000000000006,"5min":30.527999999999999,"15min":922.12},"min":{"1min":12.715,"5min":7.5830000000000002,"15min":7.3789999999999996},"max":{"1min":471.80700000000002,"5min":471.80700000000002,"15min":44816.752999999997},"last":12.154999999999999}]},{"osd":7,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":50.648000000000003,"5min":23.149999999999999,"15min":1630.123},"min":{"1min":9.8119999999999994,"5min":9.7119999999999997,"15min":7.569},"max":{"1min":564.822,"5min":564.822,"15min":65382.432000000001},"last":19.623000000000001},{"interface":"front","average":{"1min":45.387999999999998,"5min":21.542999999999999,"15min":1627.549},"min":{"1min":13.016999999999999,"5min":7.8639999999999999,"15min":7.4059999999999997},"max":{"1min":433.30799999999999,"5min":433.30799999999999,"15min":65371.873},"last":15.762}]}]},{"osd":3,"up_from":834,"seq":3582002725219,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58740,"kb_used_data":5548,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313100,"statfs":{"total":96636764160,"available":96576614400,"internally_reserved":0,"allocated":5681152,"data_stored":3019897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":9.8170000000000002,"5min":26.867000000000001,"15min":32.284999999999997},"min":{"1min":5.6029999999999998,"5min":4.8559999999999999,"15min":4.8559999999999999},"max":{"1min":18.390999999999998,"5min":188.56700000000001,"15min":188.56700000000001},"last":6.625},{"interface":"front","average":{"1min":16.754999999999999,"5min":27.206,"15min":31.43},"min":{"1min":5.9870000000000001,"5min":5.827,"15min":5.827},"max":{"1min":135.715,"5min":214.209,"15min":214.209},"last":8.5830000000000002}]},{"osd":1,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":63.271000000000001,"5min":37.026000000000003,"15min":34.704000000000001},"min":{"1min":7.7329999999999997,"5min":7.4409999999999998,"15min":7.4409999999999998},"max":{"1min":993.702,"5min":993.702,"15min":993.702},"last":10.856999999999999},{"interface":"front","average":{"1min":65.965999999999994,"5min":38.597999999999999,"15min":36.668999999999997},"min":{"1min":7.8520000000000003,"5min":6.6100000000000003,"15min":6.6100000000000003},"max":{"1min":991.50599999999997,"5min":991.50599999999997,"15min":991.50599999999997},"last":10.305}]},{"osd":2,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":21.853000000000002,"5min":29.605,"15min":34.101999999999997},"min":{"1min":8.9570000000000007,"5min":8.4199999999999999,"15min":8.4199999999999999},"max":{"1min":154.76599999999999,"5min":230.25399999999999,"15min":230.25399999999999},"last":14.02},{"interface":"front","average":{"1min":19.876000000000001,"5min":27.256,"15min":30.832000000000001},"min":{"1min":8.1959999999999997,"5min":8.1959999999999997,"15min":8.1959999999999997},"max":{"1min":95.603999999999999,"5min":165.553,"15min":165.553},"last":9.359}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":2736.415,"5min":573.53200000000004,"15min":216.17099999999999},"min":{"1min":10.223000000000001,"5min":7.1020000000000003,"15min":7.1020000000000003},"max":{"1min":40828.069000000003,"5min":40828.069000000003,"15min":40828.069000000003},"last":15.593},{"interface":"front","average":{"1min":18.699999999999999,"5min":28.707999999999998,"15min":32.601999999999997},"min":{"1min":8.6150000000000002,"5min":8.6150000000000002,"15min":8.6150000000000002},"max":{"1min":100.607,"5min":194.14599999999999,"15min":194.14599999999999},"last":11.545999999999999}]},{"osd":5,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":19.344000000000001,"5min":30.407,"15min":34.871000000000002},"min":{"1min":10.394,"5min":10.394,"15min":10.394},"max":{"1min":98.769000000000005,"5min":196.50200000000001,"15min":196.50200000000001},"last":14.930999999999999},{"interface":"front","average":{"1min":18.803000000000001,"5min":27.030999999999999,"15min":30.731000000000002},"min":{"1min":7.9329999999999998,"5min":7.9329999999999998,"15min":7.9329999999999998},"max":{"1min":111.729,"5min":177.739,"15min":177.739},"last":16.969000000000001}]},{"osd":6,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":20.492999999999999,"5min":38.835999999999999,"15min":47.399000000000001},"min":{"1min":9.6929999999999996,"5min":9.6929999999999996,"15min":9.6929999999999996},"max":{"1min":113.20399999999999,"5min":381.13,"15min":381.13},"last":17.611000000000001},{"interface":"front","average":{"1min":22.385999999999999,"5min":32.366999999999997,"15min":37.090000000000003},"min":{"1min":10.928000000000001,"5min":10.314,"15min":10.314},"max":{"1min":108.327,"5min":245.16200000000001,"15min":245.16200000000001},"last":18.335999999999999}]},{"osd":7,"last update":"Fri Oct 6 14:25:38 2023","interfaces":[{"interface":"back","average":{"1min":23.408999999999999,"5min":27.462,"15min":29.341999999999999},"min":{"1min":11.864000000000001,"5min":11.411,"15min":11.411},"max":{"1min":113.56699999999999,"5min":227.42599999999999,"15min":227.42599999999999},"last":19.111000000000001},{"interface":"front","average":{"1min":21.158000000000001,"5min":24.553999999999998,"15min":25.303999999999998},"min":{"1min":11.800000000000001,"5min":11.800000000000001,"15min":11.800000000000001},"max":{"1min":101.83499999999999,"5min":164.09899999999999,"15min":164.09899999999999},"last":13.367000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509833,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58740,"kb_used_data":5548,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313100,"statfs":{"total":96636764160,"available":96576614400,"internally_reserved":0,"allocated":5681152,"data_stored":3019897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":12.117000000000001,"5min":11.677,"15min":16.064},"min":{"1min":7.3819999999999997,"5min":5.452,"15min":5.3630000000000004},"max":{"1min":24.227,"5min":31.484000000000002,"15min":182.96600000000001},"last":22.709},{"interface":"front","average":{"1min":10.268000000000001,"5min":12.111000000000001,"15min":17.263000000000002},"min":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"max":{"1min":17.821000000000002,"5min":44.423000000000002,"15min":226.68600000000001},"last":6.6379999999999999}]},{"osd":1,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":18.187000000000001,"5min":16.309999999999999,"15min":17.614000000000001},"min":{"1min":7.6710000000000003,"5min":7.3630000000000004,"15min":6.0970000000000004},"max":{"1min":100.065,"5min":101.874,"15min":148.20400000000001},"last":22.103000000000002},{"interface":"front","average":{"1min":12.387,"5min":14.256,"15min":17.227},"min":{"1min":6.6840000000000002,"5min":6.4630000000000001,"15min":6.3140000000000001},"max":{"1min":53.959000000000003,"5min":161.97499999999999,"15min":229.82499999999999},"last":10.198}]},{"osd":3,"last update":"Fri Oct 6 14:25:48 2023","interfaces":[{"interface":"back","average":{"1min":16.808,"5min":16.744,"15min":16.396000000000001},"min":{"1min":8.2769999999999992,"5min":8.2769999999999992,"15min":8.2769999999999992},"max":{"1min":31.460000000000001,"5min":33.399999999999999,"15min":33.399999999999999},"last":21.518999999999998},{"interface":"front","average":{"1min":13.861000000000001,"5min":14.6,"15min":14.781000000000001},"min":{"1min":8.6799999999999997,"5min":8.2509999999999994,"15min":8.2509999999999994},"max":{"1min":28.713000000000001,"5min":29.898,"15min":29.898},"last":12.388999999999999}]},{"osd":4,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":14883.07,"5min":2990.154,"15min":1010.401},"min":{"1min":6.8150000000000004,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":38405.120000000003,"5min":38405.120000000003,"15min":38405.120000000003},"last":9.2859999999999996},{"interface":"front","average":{"1min":14785.589,"5min":2970.7600000000002,"15min":1006.212},"min":{"1min":7.8520000000000003,"5min":7.6550000000000002,"15min":7.6550000000000002},"max":{"1min":38264.116000000002,"5min":38264.116000000002,"15min":38264.116000000002},"last":20.184000000000001}]},{"osd":5,"last update":"Fri Oct 6 14:25:51 2023","interfaces":[{"interface":"back","average":{"1min":16.574999999999999,"5min":16.280999999999999,"15min":92.066999999999993},"min":{"1min":8.2560000000000002,"5min":8.2560000000000002,"15min":7.6589999999999998},"max":{"1min":32.328000000000003,"5min":50.210000000000001,"15min":9721.0490000000009},"last":11.539},{"interface":"front","average":{"1min":14.137,"5min":16.710999999999999,"15min":96.823999999999998},"min":{"1min":8.6099999999999994,"5min":8.6099999999999994,"15min":8.6099999999999994},"max":{"1min":24.001000000000001,"5min":73.320999999999998,"15min":9793.7890000000007},"last":8.4710000000000001}]},{"osd":6,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":14.999000000000001,"5min":17.789000000000001,"15min":1309.6320000000001},"min":{"1min":8.8689999999999998,"5min":8.8689999999999998,"15min":8.8689999999999998},"max":{"1min":23.431999999999999,"5min":44.606999999999999,"15min":44531.741000000002},"last":20.805},{"interface":"front","average":{"1min":16.434000000000001,"5min":16.972000000000001,"15min":1305.6900000000001},"min":{"1min":7.5330000000000004,"5min":7.141,"15min":7.141},"max":{"1min":49.576999999999998,"5min":49.576999999999998,"15min":44501.423000000003},"last":23.353999999999999}]},{"osd":7,"last update":"Fri Oct 6 14:25:34 2023","interfaces":[{"interface":"back","average":{"1min":20.236000000000001,"5min":20.585000000000001,"15min":1733.5730000000001},"min":{"1min":12.177,"5min":10.631,"15min":8.6189999999999998},"max":{"1min":38.399999999999999,"5min":61.658999999999999,"15min":69773.953999999998},"last":24.626999999999999},{"interface":"front","average":{"1min":21.065999999999999,"5min":18.300999999999998,"15min":1723.3030000000001},"min":{"1min":11.403,"5min":9.5909999999999993,"15min":8.5429999999999993},"max":{"1min":31.567,"5min":34.923000000000002,"15min":69547.547999999995},"last":25.547999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542532,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62840,"kb_used_data":5552,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94309000,"statfs":{"total":96636764160,"available":96572416000,"internally_reserved":0,"allocated":5685248,"data_stored":3019902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":8.6549999999999994,"5min":11.162000000000001,"15min":18.341000000000001},"min":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.4050000000000002},"max":{"1min":12.99,"5min":94.040999999999997,"15min":574.78800000000001},"last":5.0960000000000001},{"interface":"front","average":{"1min":11.332000000000001,"5min":15.573,"15min":21.341999999999999},"min":{"1min":7.5309999999999997,"5min":7.5309999999999997,"15min":4.5140000000000002},"max":{"1min":17.783999999999999,"5min":130.036,"15min":522.72699999999998},"last":8.1809999999999992}]},{"osd":1,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.231,"5min":27.030999999999999,"15min":24.16},"min":{"1min":7.3150000000000004,"5min":5.4059999999999997,"15min":5.4059999999999997},"max":{"1min":14.238,"5min":1271.6179999999999,"15min":1271.6179999999999},"last":7.2709999999999999},{"interface":"front","average":{"1min":12.869,"5min":30.803000000000001,"15min":27.870999999999999},"min":{"1min":9.9900000000000002,"5min":6.8639999999999999,"15min":6.8639999999999999},"max":{"1min":17.446999999999999,"5min":1322.71,"15min":1322.71},"last":13.134}]},{"osd":2,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.842000000000001,"5min":14.388999999999999,"15min":19.920999999999999},"min":{"1min":9.4809999999999999,"5min":6.1660000000000004,"15min":5.532},"max":{"1min":30.937999999999999,"5min":101.197,"15min":525.81600000000003},"last":14.093},{"interface":"front","average":{"1min":11.125999999999999,"5min":17.469999999999999,"15min":24.055},"min":{"1min":6.7409999999999997,"5min":6.4509999999999996,"15min":6.4509999999999996},"max":{"1min":27.02,"5min":99.492999999999995,"15min":468.70400000000001},"last":11.109999999999999}]},{"osd":3,"last update":"Fri Oct 6 14:25:41 2023","interfaces":[{"interface":"back","average":{"1min":15.800000000000001,"5min":20.021000000000001,"15min":20.113},"min":{"1min":6.8470000000000004,"5min":6.8470000000000004,"15min":6.8470000000000004},"max":{"1min":40.512,"5min":126.55500000000001,"15min":126.55500000000001},"last":13.396000000000001},{"interface":"front","average":{"1min":14.677,"5min":21.684999999999999,"15min":23.826000000000001},"min":{"1min":6.0389999999999997,"5min":6.0389999999999997,"15min":6.0389999999999997},"max":{"1min":49.473999999999997,"5min":236.27000000000001,"15min":236.27000000000001},"last":10.065}]},{"osd":4,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15821.066000000001,"5min":3179.7730000000001,"15min":1076.7719999999999},"min":{"1min":10.141,"5min":9.1189999999999998,"15min":8.7490000000000006},"max":{"1min":38720.860999999997,"5min":38720.860999999997,"15min":38720.860999999997},"last":15.917999999999999},{"interface":"front","average":{"1min":15816.687,"5min":3177.5610000000001,"15min":1074.1969999999999},"min":{"1min":6.782,"5min":6.782,"15min":6.782},"max":{"1min":38719.173000000003,"5min":38719.173000000003,"15min":38719.173000000003},"last":16.702999999999999}]},{"osd":5,"last update":"Fri Oct 6 14:25:51 2023","interfaces":[{"interface":"back","average":{"1min":17.318999999999999,"5min":21.241,"15min":107.18600000000001},"min":{"1min":8.8800000000000008,"5min":8.8800000000000008,"15min":8.8800000000000008},"max":{"1min":43.317999999999998,"5min":219.06200000000001,"15min":8032.4189999999999},"last":14.695},{"interface":"front","average":{"1min":18.963999999999999,"5min":21.207999999999998,"15min":109.759},"min":{"1min":7.4409999999999998,"5min":7.1459999999999999,"15min":7.1459999999999999},"max":{"1min":69.081000000000003,"5min":171.041,"15min":8030.5749999999998},"last":17.942}]},{"osd":7,"last update":"Fri Oct 6 14:25:37 2023","interfaces":[{"interface":"back","average":{"1min":20.959,"5min":19.791,"15min":1130.614},"min":{"1min":11.01,"5min":11.01,"15min":11.01},"max":{"1min":45.648000000000003,"5min":93.355999999999995,"15min":69750.712},"last":18.475000000000001},{"interface":"front","average":{"1min":16.279,"5min":14.369999999999999,"15min":1124.646},"min":{"1min":7.1769999999999996,"5min":6.1070000000000002,"15min":5.984},"max":{"1min":40.57,"5min":121.146,"15min":69749.794999999998},"last":9.1099999999999994}]}]},{"osd":5,"up_from":659,"seq":2830383448441,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77180,"kb_used_data":5940,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94294660,"statfs":{"total":96636764160,"available":96557731840,"internally_reserved":0,"allocated":6082560,"data_stored":3417737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":10.645,"5min":13.339,"15min":38.990000000000002},"min":{"1min":4.6189999999999998,"5min":4.3339999999999996,"15min":4.1879999999999997},"max":{"1min":22.547999999999998,"5min":173.54599999999999,"15min":680.28200000000004},"last":9.4489999999999998},{"interface":"front","average":{"1min":7.4779999999999998,"5min":11.548999999999999,"15min":29.652000000000001},"min":{"1min":4.7430000000000003,"5min":4.5579999999999998,"15min":4.5579999999999998},"max":{"1min":16.622,"5min":154.36199999999999,"15min":608.46199999999999},"last":7.4889999999999999}]},{"osd":1,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":45.543999999999997,"5min":20.469000000000001,"15min":43.674999999999997},"min":{"1min":6.3620000000000001,"5min":5.4930000000000003,"15min":5.4930000000000003},"max":{"1min":633.78700000000003,"5min":633.78700000000003,"15min":633.78700000000003},"last":5.1539999999999999},{"interface":"front","average":{"1min":46.32,"5min":23.73,"15min":46.381},"min":{"1min":10.032,"5min":5.9000000000000004,"15min":5.8259999999999996},"max":{"1min":598.60000000000002,"5min":598.60000000000002,"15min":701.19399999999996},"last":11.856999999999999}]},{"osd":2,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":14.177,"5min":14.871,"15min":44.731999999999999},"min":{"1min":8.218,"5min":6.8369999999999997,"15min":6.6029999999999998},"max":{"1min":27.632000000000001,"5min":87.468000000000004,"15min":603.024},"last":8.7620000000000005},{"interface":"front","average":{"1min":15.343,"5min":17.352,"15min":49.799999999999997},"min":{"1min":10.210000000000001,"5min":8.1289999999999996,"15min":8.1289999999999996},"max":{"1min":29.491,"5min":84.820999999999998,"15min":690.71799999999996},"last":14.759}]},{"osd":3,"last update":"Fri Oct 6 14:25:42 2023","interfaces":[{"interface":"back","average":{"1min":15.289999999999999,"5min":28.317,"15min":32.777999999999999},"min":{"1min":9.0920000000000005,"5min":8.1690000000000005,"15min":8.1690000000000005},"max":{"1min":31.006,"5min":264.37900000000002,"15min":264.37900000000002},"last":14.089},{"interface":"front","average":{"1min":16.370000000000001,"5min":27.132999999999999,"15min":30.166},"min":{"1min":11.48,"5min":9.0990000000000002,"15min":9.0990000000000002},"max":{"1min":33.726999999999997,"5min":173.858,"15min":173.858},"last":16.445}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":3491.913,"5min":712.81600000000003,"15min":274.36799999999999},"min":{"1min":10.141,"5min":7.9340000000000002,"15min":7.9340000000000002},"max":{"1min":38256.345000000001,"5min":38256.345000000001,"15min":38256.345000000001},"last":18.117000000000001},{"interface":"front","average":{"1min":3489.7570000000001,"5min":714.00800000000004,"15min":281.79899999999998},"min":{"1min":13.077999999999999,"5min":9.1750000000000007,"15min":8.1839999999999993},"max":{"1min":38222.745999999999,"5min":38222.745999999999,"15min":38222.745999999999},"last":19.027000000000001}]},{"osd":6,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":15.797000000000001,"5min":18.550999999999998,"15min":47.345999999999997},"min":{"1min":10.766,"5min":7.7300000000000004,"15min":7.7300000000000004},"max":{"1min":27.385000000000002,"5min":108.053,"15min":607.87400000000002},"last":16.905999999999999},{"interface":"front","average":{"1min":17.838000000000001,"5min":21.890000000000001,"15min":51.551000000000002},"min":{"1min":14.509,"5min":10.855,"15min":7.1040000000000001},"max":{"1min":26.152000000000001,"5min":91.733000000000004,"15min":739.33100000000002},"last":19.920999999999999}]},{"osd":7,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":17.960999999999999,"5min":21.77,"15min":47.536000000000001},"min":{"1min":10.858000000000001,"5min":9.3569999999999993,"15min":7.0350000000000001},"max":{"1min":32.581000000000003,"5min":90.037000000000006,"15min":609.59799999999996},"last":17.545999999999999},{"interface":"front","average":{"1min":9.0069999999999997,"5min":13.292,"15min":44.540999999999997},"min":{"1min":6.4420000000000002,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":13.343999999999999,"5min":77.034999999999997,"15min":610.43799999999999},"last":10.061}]}]},{"osd":7,"up_from":13,"seq":55834575235,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80884,"kb_used_data":5548,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94290956,"statfs":{"total":96636764160,"available":96553938944,"internally_reserved":0,"allocated":5681152,"data_stored":3019897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:31 2023","interfaces":[{"interface":"back","average":{"1min":8.9030000000000005,"5min":10.318,"15min":32.875999999999998},"min":{"1min":4.8620000000000001,"5min":4.742,"15min":4.2539999999999996},"max":{"1min":23.603000000000002,"5min":46.529000000000003,"15min":930.62099999999998},"last":4.8620000000000001},{"interface":"front","average":{"1min":11.708,"5min":11.972,"15min":31.646999999999998},"min":{"1min":5.1619999999999999,"5min":4.8079999999999998,"15min":4.8079999999999998},"max":{"1min":39.262,"5min":69.477999999999994,"15min":879.35199999999998},"last":8.6329999999999991}]},{"osd":1,"last update":"Fri Oct 6 14:26:31 2023","interfaces":[{"interface":"back","average":{"1min":11.952999999999999,"5min":26.451000000000001,"15min":39.228000000000002},"min":{"1min":6.9059999999999997,"5min":5.4790000000000001,"15min":5.4790000000000001},"max":{"1min":23.271000000000001,"5min":1094.5889999999999,"15min":1094.5889999999999},"last":9.6519999999999992},{"interface":"front","average":{"1min":12.124000000000001,"5min":25.449000000000002,"15min":24.937000000000001},"min":{"1min":6.2110000000000003,"5min":6.2110000000000003,"15min":5.4539999999999997},"max":{"1min":28.507999999999999,"5min":1057.856,"15min":1057.856},"last":11.548}]},{"osd":2,"last update":"Fri Oct 6 14:26:31 2023","interfaces":[{"interface":"back","average":{"1min":11.727,"5min":15.356,"15min":23.838000000000001},"min":{"1min":6.9969999999999999,"5min":6.5149999999999997,"15min":6.202},"max":{"1min":30.532,"5min":75.823999999999998,"15min":316.84399999999999},"last":12.172000000000001},{"interface":"front","average":{"1min":14.474,"5min":16.344000000000001,"15min":39.460000000000001},"min":{"1min":8.4879999999999995,"5min":5.6500000000000004,"15min":5.6500000000000004},"max":{"1min":31.562999999999999,"5min":98.183000000000007,"15min":931.94500000000005},"last":17.390999999999998}]},{"osd":3,"last update":"Fri Oct 6 14:25:47 2023","interfaces":[{"interface":"back","average":{"1min":14.034000000000001,"5min":20.844000000000001,"15min":22.948},"min":{"1min":6.9969999999999999,"5min":6.9969999999999999,"15min":6.9969999999999999},"max":{"1min":40.936,"5min":184.93700000000001,"15min":184.93700000000001},"last":13.16},{"interface":"front","average":{"1min":13.778,"5min":18.068999999999999,"15min":18.817},"min":{"1min":8.2010000000000005,"5min":7.6699999999999999,"15min":7.6699999999999999},"max":{"1min":32.956000000000003,"5min":77.144000000000005,"15min":77.144000000000005},"last":19.154}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":2132.1480000000001,"5min":439.87099999999998,"15min":167.66999999999999},"min":{"1min":7.8869999999999996,"5min":5.9429999999999996,"15min":5.9429999999999996},"max":{"1min":40210.642,"5min":40210.642,"15min":40210.642},"last":12.739000000000001},{"interface":"front","average":{"1min":2144.9340000000002,"5min":442.64400000000001,"15min":163.51599999999999},"min":{"1min":11.497999999999999,"5min":7.4180000000000001,"15min":6.9279999999999999},"max":{"1min":40377.220000000001,"5min":40377.220000000001,"15min":40377.220000000001},"last":15.577999999999999}]},{"osd":5,"last update":"Fri Oct 6 14:25:50 2023","interfaces":[{"interface":"back","average":{"1min":17.524000000000001,"5min":18.869,"15min":51.777000000000001},"min":{"1min":8.4939999999999998,"5min":8.1229999999999993,"15min":7.6909999999999998},"max":{"1min":70.796000000000006,"5min":149.61799999999999,"15min":6750.3770000000004},"last":13.901999999999999},{"interface":"front","average":{"1min":19.053999999999998,"5min":19.542000000000002,"15min":52.655000000000001},"min":{"1min":7.9029999999999996,"5min":6.9290000000000003,"15min":6.9290000000000003},"max":{"1min":83.019000000000005,"5min":174.839,"15min":6875.5780000000004},"last":14.23}]},{"osd":6,"last update":"Fri Oct 6 14:26:31 2023","interfaces":[{"interface":"back","average":{"1min":15.701000000000001,"5min":18.128,"15min":26.704000000000001},"min":{"1min":10.023,"5min":7.9509999999999996,"15min":7.7290000000000001},"max":{"1min":36.390000000000001,"5min":75.013999999999996,"15min":380.74000000000001},"last":14.987},{"interface":"front","average":{"1min":18.978000000000002,"5min":20.349,"15min":44.131},"min":{"1min":9.2360000000000007,"5min":8.9489999999999998,"15min":8.9489999999999998},"max":{"1min":82.266999999999996,"5min":120.83799999999999,"15min":935.49300000000005},"last":18.279}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T14:26:36.218 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.218 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.218 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.219 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.219 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.219 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.219 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.219 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.219 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.219 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.220 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.220 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.220 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.220 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-06T13:53:14.910188+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=53, tm_sec=14, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.220 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.220 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-06T13:53:14.910188+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=53, tm_sec=14, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.220 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.221 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-06T13:53:14.910188+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=53, tm_sec=14, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.221 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.221 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-06T13:53:14.910188+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=53, tm_sec=14, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.221 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.221 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-06T13:53:14.910188+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=53, tm_sec=14, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.221 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.221 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-06T13:53:14.910188+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=53, tm_sec=14, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.222 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.222 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T13:53:14.910188+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=53, tm_sec=14, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.222 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-06T13:53:14.910188+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=53, tm_sec=14, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.222 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.222 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.222 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.223 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.223 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.223 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.223 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.223 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.223 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.224 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.224 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:36.224 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T14:26:38.510 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:41.633 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:38.513 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:41.636 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:38.527 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:41.650 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:38.559 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:41.656 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:38.560 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:41.657 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:38.562 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:41.685 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:38.565 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:41.688 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:38.571 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:41.668 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:38.578 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:41.701 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:43.524 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:46.646 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:43.528 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:46.651 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:43.540 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:46.663 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:43.572 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:46.668 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:43.573 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:46.670 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:43.578 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:46.701 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:43.581 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:46.704 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:43.585 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:46.682 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:43.592 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:46.715 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:53.575 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:56.698 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:53.600 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:56.699 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:53.600 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:34:56.714 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:53.622 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:56.744 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:53.794 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:56.746 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:53.794 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:34:56.760 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:53.795 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:56.724 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:53.795 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:56.725 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:53.795 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:34:56.740 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:26:56.225 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T14:26:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.290+0000 7f6dbd296640 10 monclient: get_monmap_and_config 2023-10-06T14:26:57.833 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.290+0000 7f6dbd296640 10 monclient: build_initial_monmap 2023-10-06T14:26:57.833 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.290+0000 7f6dbd296640 10 monclient: monmap: 2023-10-06T14:26:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:26:57.291507+0000 2023-10-06T14:26:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:26:57.291507+0000 2023-10-06T14:26:57.835 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:57.835 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:57.835 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:26:57.835 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:26:57.836 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:26:57.836 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:26:57.836 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:26:57.836 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:57.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 1 Processor -- start 2023-10-06T14:26:57.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 1 -- start start 2023-10-06T14:26:57.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 10 monclient: init 2023-10-06T14:26:57.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:57.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2023-10-06T14:26:57.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6db80b3300 addr v1:172.21.15.83:6789/0 2023-10-06T14:26:57.851 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6db8070e20 0x7f6db8140e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 10 monclient(hunting): picked mon.noname-c con 0x7f6db8070e20 addr v2:172.21.15.83:3300/0 2023-10-06T14:26:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6db813abf0 addr v1:172.21.15.19:6789/0 2023-10-06T14:26:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db80abe50 con 0x7f6db813abf0 2023-10-06T14:26:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db80ab5c0 con 0x7f6db80b3300 2023-10-06T14:26:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db80ab8a0 con 0x7f6db8070e20 2023-10-06T14:26:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:57.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.291+0000 7f6dbd296640 10 monclient(hunting): authenticate will time out at 3345.478067s 2023-10-06T14:26:57.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.292+0000 7f6dae7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6db8070e20 0x7f6db8140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:57.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.292+0000 7f6db77fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6db813abf0 0x7f6db813b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36224/0 (socket says 172.21.15.19:36224) 2023-10-06T14:26:57.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.292+0000 7f6db77fe640 1 -- 172.21.15.19:0/3623120976 learned_addr learned my addr 172.21.15.19:0/3623120976 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T14:26:57.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.292+0000 7f6dae7fc640 10 monclient(hunting): get_auth_request con 0x7f6db8070e20 auth_method 0 2023-10-06T14:26:57.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.292+0000 7f6dae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:57.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.292+0000 7f6dae7fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:57.853 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.292+0000 7f6dae7fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:57.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.293+0000 7f6dae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:57.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.293+0000 7f6dae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:57.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.293+0000 7f6dae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:57.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.294+0000 7f6db67fc640 1 -- 172.21.15.19:0/3623120976 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753920008 0 0) 0x7f6db80abe50 con 0x7f6db813abf0 2023-10-06T14:26:57.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.294+0000 7f6db67fc640 10 monclient(hunting): _negotiate 2023-10-06T14:26:57.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.294+0000 7f6db67fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:57.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.294+0000 7f6db67fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:57.854 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.294+0000 7f6db67fc640 10 monclient(hunting): my global_id is 15711 2023-10-06T14:26:57.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.294+0000 7f6db67fc640 1 -- 172.21.15.19:0/3623120976 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6da4005730 con 0x7f6db813abf0 2023-10-06T14:26:57.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.295+0000 7f6dae7fc640 10 monclient(hunting): handle_auth_done global_id 20827 payload 274 2023-10-06T14:26:57.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.295+0000 7f6dae7fc640 1 -- 172.21.15.19:0/3623120976 >> v1:172.21.15.83:6789/0 conn(0x7f6db80b3300 legacy=0x7f6db80ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:26:57.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.295+0000 7f6dae7fc640 1 -- 172.21.15.19:0/3623120976 >> v1:172.21.15.19:6789/0 conn(0x7f6db813abf0 legacy=0x7f6db813b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:57.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.295+0000 7f6dae7fc640 10 monclient: _finish_hunting 0 2023-10-06T14:26:57.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.295+0000 7f6dae7fc640 1 monclient: found mon.noname-c 2023-10-06T14:26:57.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.295+0000 7f6dae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:57.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.295+0000 7f6dae7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.83:3300/0 2023-10-06T14:26:57.856 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.295+0000 7f6dae7fc640 1 -- 172.21.15.19:0/3623120976 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db80718c0 con 0x7f6db8070e20 2023-10-06T14:26:57.860 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.298+0000 7f6dae7fc640 1 --2- 172.21.15.19:0/3623120976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6db8070e20 0x7f6db8140e40 secure :-1 s=READY pgs=5135 cs=0 l=1 rev1=1 crypto rx=0x7f6da00097a0 tx=0x7f6da002b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c054546ac308a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:57.860 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6db67fc640 1 -- 172.21.15.19:0/3623120976 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da002f050 con 0x7f6db8070e20 2023-10-06T14:26:57.860 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6db67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:57.860 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6db67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-10-06T14:26:57.860 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6db67fc640 10 monclient: dump: 2023-10-06T14:26:57.860 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:57.862 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:57.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6db67fc640 10 monclient: _finish_auth 0 2023-10-06T14:26:57.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6db67fc640 10 monclient: _check_auth_tickets 2023-10-06T14:26:57.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6db67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:57.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6db67fc640 1 -- 172.21.15.19:0/3623120976 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da0031700 con 0x7f6db8070e20 2023-10-06T14:26:57.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6dbd296640 5 monclient: authenticate success, global_id 20827 2023-10-06T14:26:57.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6dbd296640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-10-06T14:26:57.862 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6db67fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:57.863 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6dbd296640 10 monclient: get_monmap_and_config success 2023-10-06T14:26:57.863 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6db67fc640 1 -- 172.21.15.19:0/3623120976 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0031a90 con 0x7f6db8070e20 2023-10-06T14:26:57.863 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6dbd296640 4 set_mon_vals no callback set 2023-10-06T14:26:57.863 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6db67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:57.863 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6db67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:57.863 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.302+0000 7f6db67fc640 10 monclient: dump: 2023-10-06T14:26:57.863 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:57.864 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:57.864 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:57.864 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:57.864 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:57.864 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:57.864 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:57.864 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:57.864 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:57.865 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:57.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.303+0000 7f6dbd296640 10 monclient: shutdown 2023-10-06T14:26:57.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.303+0000 7f6dbd296640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:57.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.303+0000 7f6dbd296640 1 -- 172.21.15.19:0/3623120976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6db8070e20 msgr2=0x7f6db8140e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:57.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.303+0000 7f6dbd296640 1 --2- 172.21.15.19:0/3623120976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6db8070e20 0x7f6db8140e40 secure :-1 s=READY pgs=5135 cs=0 l=1 rev1=1 crypto rx=0x7f6da00097a0 tx=0x7f6da002b7a0 comp rx=0 tx=0).stop 2023-10-06T14:26:57.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.303+0000 7f6dbd296640 1 -- 172.21.15.19:0/3623120976 shutdown_connections 2023-10-06T14:26:57.865 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.303+0000 7f6dbd296640 1 --2- 172.21.15.19:0/3623120976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6db8070e20 0x7f6db8140e40 unknown :-1 s=CLOSED pgs=5135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:57.866 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.303+0000 7f6dbd296640 1 -- 172.21.15.19:0/3623120976 >> 172.21.15.19:0/3623120976 conn(0x7f6db8065f50 msgr2=0x7f6db80b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:57.866 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.303+0000 7f6dbd296640 1 -- 172.21.15.19:0/3623120976 shutdown_connections 2023-10-06T14:26:57.866 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.303+0000 7f6dbd296640 1 -- 172.21.15.19:0/3623120976 wait complete. 2023-10-06T14:26:57.866 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.304+0000 7f6dbd296640 10 monclient: build_initial_monmap 2023-10-06T14:26:57.866 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.304+0000 7f6dbd296640 10 monclient: monmap: 2023-10-06T14:26:57.866 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:26:57.866 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:26:57.867 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:26:57.867 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:26:57.867 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:26:57.867 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:57.867 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:26:57.867 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:26:57.867 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:26:57.868 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:57.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.304+0000 7f6dbd296640 1 librados: starting msgr at 2023-10-06T14:26:57.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.304+0000 7f6dbd296640 1 librados: starting objecter 2023-10-06T14:26:57.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.304+0000 7f6dbd296640 1 Processor -- start 2023-10-06T14:26:57.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.304+0000 7f6dbd296640 1 -- start start 2023-10-06T14:26:57.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.304+0000 7f6dbd296640 1 librados: setting wanted keys 2023-10-06T14:26:57.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.304+0000 7f6dbd296640 1 librados: calling monclient init 2023-10-06T14:26:57.868 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.304+0000 7f6dbd296640 10 monclient: init 2023-10-06T14:26:57.869 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 10 monclient: _reopen_session rank -1 2023-10-06T14:26:57.869 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 10 monclient: _add_conns ranks=[1,2,0] 2023-10-06T14:26:57.869 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6db8070e20 0x7f6db813d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:57.869 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 10 monclient(hunting): picked mon.noname-b con 0x7f6db8070e20 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:26:57.869 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f6db80b3300 0x7f6db81d9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:57.869 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 10 monclient(hunting): picked mon.noname-c con 0x7f6db80b3300 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:26:57.869 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db813abf0 0x7f6db8137d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:57.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 10 monclient(hunting): picked mon.noname-a con 0x7f6db813abf0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:26:57.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:57.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db80709b0 con 0x7f6db813abf0 2023-10-06T14:26:57.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:57.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db81d35f0 con 0x7f6db8070e20 2023-10-06T14:26:57.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 10 monclient(hunting): start opening mon connection 2023-10-06T14:26:57.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db81d9e80 con 0x7f6db80b3300 2023-10-06T14:26:57.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 10 monclient(hunting): _renew_subs 2023-10-06T14:26:57.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dbd296640 10 monclient(hunting): authenticate will time out at 3345.491674s 2023-10-06T14:26:57.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6db77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db813abf0 0x7f6db8137d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:57.870 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6dae7fc640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f6db80b3300 0x7f6db81d9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.305+0000 7f6db6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6db8070e20 0x7f6db813d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6dae7fc640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f6db80b3300 0x7f6db81d9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:52626/0 (socket says 172.21.15.19:52626) 2023-10-06T14:26:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6dae7fc640 1 -- 172.21.15.19:0/333391002 learned_addr learned my addr 172.21.15.19:0/333391002 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:26:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6dae7fc640 10 monclient(hunting): get_auth_request con 0x7f6db80b3300 auth_method 0 2023-10-06T14:26:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6dae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6dae7fc640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6dae7fc640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6db6ffd640 10 monclient(hunting): get_auth_request con 0x7f6db8070e20 auth_method 0 2023-10-06T14:26:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6db6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6db6ffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6db6ffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6db77fe640 10 monclient(hunting): get_auth_request con 0x7f6db813abf0 auth_method 0 2023-10-06T14:26:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6db77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:26:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6db77fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:26:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6db77fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:26:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6db6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6db6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6db6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6dae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6dae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6dae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6db77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:26:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6db77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:26:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.306+0000 7f6db77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:26:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.309+0000 7f6db6ffd640 10 monclient(hunting): handle_auth_done global_id 20830 payload 824 2023-10-06T14:26:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.309+0000 7f6db6ffd640 1 -- 172.21.15.19:0/333391002 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f6db80b3300 msgr2=0x7f6db81d9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.309+0000 7f6db6ffd640 1 --2- 172.21.15.19:0/333391002 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f6db80b3300 0x7f6db81d9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.309+0000 7f6db6ffd640 1 -- 172.21.15.19:0/333391002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db813abf0 msgr2=0x7f6db8137d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.309+0000 7f6db6ffd640 1 --2- 172.21.15.19:0/333391002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db813abf0 0x7f6db8137d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.309+0000 7f6db6ffd640 10 monclient: _finish_hunting 0 2023-10-06T14:26:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.309+0000 7f6db6ffd640 1 monclient: found mon.noname-b 2023-10-06T14:26:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.309+0000 7f6db6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:26:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.309+0000 7f6db6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T14:26:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.309+0000 7f6db6ffd640 1 -- 172.21.15.19:0/333391002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da0009400 con 0x7f6db8070e20 2023-10-06T14:26:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.311+0000 7f6db6ffd640 1 --2- 172.21.15.19:0/333391002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6db8070e20 0x7f6db813d3b0 secure :-1 s=READY pgs=5136 cs=0 l=1 rev1=1 crypto rx=0x7f6da800c780 tx=0x7f6da800cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.311+0000 7f6daffff640 10 client.?.objecter ms_handle_connect 0x7f6db8070e20 2023-10-06T14:26:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.311+0000 7f6daffff640 10 client.?.objecter resend_mon_ops 2023-10-06T14:26:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6daffff640 1 -- 172.21.15.19:0/333391002 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800aa50 con 0x7f6db8070e20 2023-10-06T14:26:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6daffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6daffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:26:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6daffff640 10 monclient: dump: 2023-10-06T14:26:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:57.875 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6daffff640 10 monclient: _finish_auth 0 2023-10-06T14:26:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6daffff640 10 monclient: _check_auth_tickets 2023-10-06T14:26:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6daffff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:26:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6daffff640 1 -- 172.21.15.19:0/333391002 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da800abf0 con 0x7f6db8070e20 2023-10-06T14:26:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6dbd296640 5 monclient: authenticate success, global_id 20830 2023-10-06T14:26:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6dbd296640 10 monclient: _renew_subs 2023-10-06T14:26:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6dbd296640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6dbd296640 1 -- 172.21.15.19:0/333391002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db81d9b80 con 0x7f6db8070e20 2023-10-06T14:26:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6daffff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:26:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6daffff640 1 -- 172.21.15.19:0/333391002 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800ce30 con 0x7f6db8070e20 2023-10-06T14:26:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6daffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:26:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6daffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:26:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6daffff640 10 monclient: dump: 2023-10-06T14:26:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:26:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:26:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:26:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:26:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:26:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:26:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:26:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:26:57.878 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:26:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6db5ffb640 4 set_mon_vals no callback set 2023-10-06T14:26:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6dbd296640 10 client.20830.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:26:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6dbd296640 10 monclient: _renew_subs 2023-10-06T14:26:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6dbd296640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6dbd296640 1 -- 172.21.15.19:0/333391002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db81382b0 con 0x7f6db8070e20 2023-10-06T14:26:57.879 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.315+0000 7f6dbd296640 1 librados: init done 2023-10-06T14:26:57.879 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.316+0000 7f6dbd296640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:57.879 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.316+0000 7f6dbd296640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:57.879 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.316+0000 7f6dbd296640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:26:57.879 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.316+0000 7f6dbd296640 1 -- 172.21.15.19:0/333391002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db81da0a0 con 0x7f6db8070e20 2023-10-06T14:26:57.879 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.335+0000 7f6daffff640 1 -- 172.21.15.19:0/333391002 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f6da8024430 con 0x7f6db8070e20 2023-10-06T14:26:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.335+0000 7f6daffff640 1 --2- 172.21.15.19:0/333391002 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f6d9806c1c0 0x7f6d9806e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:26:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.335+0000 7f6dae7fc640 1 --2- 172.21.15.19:0/333391002 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f6d9806c1c0 0x7f6d9806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:26:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.335+0000 7f6dae7fc640 10 monclient: get_auth_request con 0x7f6d9806c1c0 auth_method 0 2023-10-06T14:26:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.336+0000 7f6dae7fc640 1 --2- 172.21.15.19:0/333391002 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f6d9806c1c0 0x7f6d9806e670 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6db81d39d0 tx=0x7f6da0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:26:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.336+0000 7f6daffff640 10 client.20830.objecter ms_handle_connect 0x7f6d9806c1c0 2023-10-06T14:26:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.337+0000 7f6daffff640 1 -- 172.21.15.19:0/333391002 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(970..970 src has 714..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f6da8058460 con 0x7f6db8070e20 2023-10-06T14:26:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.337+0000 7f6daffff640 10 client.20830.objecter ms_dispatch 0x7f6db8000b90 osd_map(970..970 src has 714..970) v4 2023-10-06T14:26:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.337+0000 7f6daffff640 3 client.20830.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:26:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.337+0000 7f6daffff640 3 client.20830.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:26:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.337+0000 7f6daffff640 20 client.20830.objecter dump_active .. 0 homeless 2023-10-06T14:26:57.881 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.564+0000 7f6daffff640 1 -- 172.21.15.19:0/333391002 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f6da80547a0 con 0x7f6db8070e20 2023-10-06T14:26:57.881 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.564+0000 7f6daffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:26:57.881 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.564+0000 7f6daffff640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:26:57.881 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.737+0000 7f6dbd296640 1 -- 172.21.15.19:0/333391002 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6db81d9d10 con 0x7f6d9806c1c0 2023-10-06T14:26:57.881 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.743+0000 7f6daffff640 1 -- 172.21.15.19:0/333391002 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152342 (secure 0 0 0) 0x7f6db81d9d10 con 0x7f6d9806c1c0 2023-10-06T14:26:57.883 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T14:26:57.884 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2023-10-06T14:26:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 10 librados: watch_flush enter 2023-10-06T14:26:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 10 librados: watch_flush exit 2023-10-06T14:26:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 20 client.20830.objecter shutdown clearing up homeless session... 2023-10-06T14:26:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 10 client.20830.objecter successfully canceled tick 2023-10-06T14:26:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 1 -- 172.21.15.19:0/333391002 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f6d9806c1c0 msgr2=0x7f6d9806e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 1 --2- 172.21.15.19:0/333391002 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f6d9806c1c0 0x7f6d9806e670 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6db81d39d0 tx=0x7f6da0008030 comp rx=0 tx=0).stop 2023-10-06T14:26:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 10 monclient: shutdown 2023-10-06T14:26:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:26:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 1 -- 172.21.15.19:0/333391002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6db8070e20 msgr2=0x7f6db813d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:26:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 1 --2- 172.21.15.19:0/333391002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6db8070e20 0x7f6db813d3b0 secure :-1 s=READY pgs=5136 cs=0 l=1 rev1=1 crypto rx=0x7f6da800c780 tx=0x7f6da800cc40 comp rx=0 tx=0).stop 2023-10-06T14:26:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 1 -- 172.21.15.19:0/333391002 shutdown_connections 2023-10-06T14:26:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 1 --2- 172.21.15.19:0/333391002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db813abf0 0x7f6db8137d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 1 --2- 172.21.15.19:0/333391002 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f6d9806c1c0 0x7f6d9806e670 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 1 --2- 172.21.15.19:0/333391002 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f6db80b3300 0x7f6db81d9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 1 --2- 172.21.15.19:0/333391002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6db8070e20 0x7f6db813d3b0 unknown :-1 s=CLOSED pgs=5136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:26:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 1 -- 172.21.15.19:0/333391002 >> 172.21.15.19:0/333391002 conn(0x7f6db8065f50 msgr2=0x7f6db80b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:26:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 1 -- 172.21.15.19:0/333391002 shutdown_connections 2023-10-06T14:26:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 1 -- 172.21.15.19:0/333391002 wait complete. 2023-10-06T14:26:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:26:57.745+0000 7f6dbd296640 1 librados: shutdown 2023-10-06T14:26:57.888 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":1988,"stamp":"2023-10-06T14:26:55.810793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":515000,"kb_used_data":45176,"kb_used_omap":0,"kb_used_meta":469760,"kb_avail":754459720,"statfs":{"total":773094113280,"available":772566753280,"internally_reserved":0,"allocated":46260224,"data_stored":24954866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":481034240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002678"},"pg_stats":[{"pgid":"8.14","version":"0'0","reported_seq":935,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:36.503889+0000","last_change":"2023-10-06T14:26:36.505925+0000","last_active":"2023-10-06T14:26:36.503889+0000","last_peered":"2023-10-06T14:26:36.503889+0000","last_clean":"2023-10-06T14:26:36.503889+0000","last_became_active":"2023-10-06T13:54:27.664679+0000","last_became_peered":"2023-10-06T13:54:27.664679+0000","last_unstale":"2023-10-06T14:26:36.503889+0000","last_undegraded":"2023-10-06T14:26:36.503889+0000","last_fullsized":"2023-10-06T14:26:36.503889+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:36.502985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:36.502985+0000","last_clean_scrub_stamp":"2023-10-06T14:26:36.502985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:28.334974+0000","scrub_duration":0.037170340000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":923,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.379601+0000","last_change":"2023-10-06T14:22:39.751625+0000","last_active":"2023-10-06T14:26:28.379601+0000","last_peered":"2023-10-06T14:26:28.379601+0000","last_clean":"2023-10-06T14:26:28.379601+0000","last_became_active":"2023-10-06T14:22:39.724017+0000","last_became_peered":"2023-10-06T14:22:39.724017+0000","last_unstale":"2023-10-06T14:26:28.379601+0000","last_undegraded":"2023-10-06T14:26:28.379601+0000","last_fullsized":"2023-10-06T14:26:28.379601+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":939,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:47.942073+0000","last_change":"2023-10-06T14:26:47.947183+0000","last_active":"2023-10-06T14:26:47.942073+0000","last_peered":"2023-10-06T14:26:47.942073+0000","last_clean":"2023-10-06T14:26:47.942073+0000","last_became_active":"2023-10-06T14:16:33.542923+0000","last_became_peered":"2023-10-06T14:16:33.542923+0000","last_unstale":"2023-10-06T14:26:47.942073+0000","last_undegraded":"2023-10-06T14:26:47.942073+0000","last_fullsized":"2023-10-06T14:26:47.942073+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:47.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:47.941299+0000","last_clean_scrub_stamp":"2023-10-06T14:26:47.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:42:17.558417+0000","scrub_duration":0.046222051,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":941,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:47.724203+0000","last_change":"2023-10-06T14:26:47.729390+0000","last_active":"2023-10-06T14:26:47.724203+0000","last_peered":"2023-10-06T14:26:47.724203+0000","last_clean":"2023-10-06T14:26:47.724203+0000","last_became_active":"2023-10-06T13:54:29.163161+0000","last_became_peered":"2023-10-06T13:54:29.163161+0000","last_unstale":"2023-10-06T14:26:47.724203+0000","last_undegraded":"2023-10-06T14:26:47.724203+0000","last_fullsized":"2023-10-06T14:26:47.724203+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:47.723383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:47.723383+0000","last_clean_scrub_stamp":"2023-10-06T14:26:47.723383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:06:07.573686+0000","scrub_duration":0.070208072999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":922,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.353597+0000","last_change":"2023-10-06T14:22:39.220927+0000","last_active":"2023-10-06T14:26:28.353597+0000","last_peered":"2023-10-06T14:26:28.353597+0000","last_clean":"2023-10-06T14:26:28.353597+0000","last_became_active":"2023-10-06T14:22:39.128927+0000","last_became_peered":"2023-10-06T14:22:39.128927+0000","last_unstale":"2023-10-06T14:26:28.353597+0000","last_undegraded":"2023-10-06T14:26:28.353597+0000","last_fullsized":"2023-10-06T14:26:28.353597+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:54:15.007539+0000","last_clean_scrub_stamp":"2023-10-06T13:54:15.007539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":934,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:39.527703+0000","last_change":"2023-10-06T14:26:39.533028+0000","last_active":"2023-10-06T14:26:39.527703+0000","last_peered":"2023-10-06T14:26:39.527703+0000","last_clean":"2023-10-06T14:26:39.527703+0000","last_became_active":"2023-10-06T14:22:39.418221+0000","last_became_peered":"2023-10-06T14:22:39.418221+0000","last_unstale":"2023-10-06T14:26:39.527703+0000","last_undegraded":"2023-10-06T14:26:39.527703+0000","last_fullsized":"2023-10-06T14:26:39.527703+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:39.526884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:39.526884+0000","last_clean_scrub_stamp":"2023-10-06T14:26:39.526884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:54:10.734371+0000","scrub_duration":0.069040852,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":934,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:49.532283+0000","last_change":"2023-10-06T14:26:49.534206+0000","last_active":"2023-10-06T14:26:49.532283+0000","last_peered":"2023-10-06T14:26:49.532283+0000","last_clean":"2023-10-06T14:26:49.532283+0000","last_became_active":"2023-10-06T14:22:39.195040+0000","last_became_peered":"2023-10-06T14:22:39.195040+0000","last_unstale":"2023-10-06T14:26:49.532283+0000","last_undegraded":"2023-10-06T14:26:49.532283+0000","last_fullsized":"2023-10-06T14:26:49.532283+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:49.531420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:49.531420+0000","last_clean_scrub_stamp":"2023-10-06T14:26:49.531420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:38.168606+0000","scrub_duration":0.058446350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":940,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:44.019889+0000","last_change":"2023-10-06T14:26:44.021646+0000","last_active":"2023-10-06T14:26:44.019889+0000","last_peered":"2023-10-06T14:26:44.019889+0000","last_clean":"2023-10-06T14:26:44.019889+0000","last_became_active":"2023-10-06T14:16:33.300222+0000","last_became_peered":"2023-10-06T14:16:33.300222+0000","last_unstale":"2023-10-06T14:26:44.019889+0000","last_undegraded":"2023-10-06T14:26:44.019889+0000","last_fullsized":"2023-10-06T14:26:44.019889+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:44.019035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:44.019035+0000","last_clean_scrub_stamp":"2023-10-06T14:26:44.019035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:08.569157+0000","scrub_duration":0.061160045000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":933,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:50.531352+0000","last_change":"2023-10-06T14:26:50.533277+0000","last_active":"2023-10-06T14:26:50.531352+0000","last_peered":"2023-10-06T14:26:50.531352+0000","last_clean":"2023-10-06T14:26:50.531352+0000","last_became_active":"2023-10-06T14:22:39.712413+0000","last_became_peered":"2023-10-06T14:22:39.712413+0000","last_unstale":"2023-10-06T14:26:50.531352+0000","last_undegraded":"2023-10-06T14:26:50.531352+0000","last_fullsized":"2023-10-06T14:26:50.531352+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:50.530490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:50.530490+0000","last_clean_scrub_stamp":"2023-10-06T14:26:50.530490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:59.318217+0000","scrub_duration":0.045610221999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":922,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:43.874971+0000","last_change":"2023-10-06T14:26:43.874971+0000","last_active":"2023-10-06T14:26:43.874971+0000","last_peered":"2023-10-06T14:26:43.874971+0000","last_clean":"2023-10-06T14:26:43.874971+0000","last_became_active":"2023-10-06T13:54:19.427005+0000","last_became_peered":"2023-10-06T13:54:19.427005+0000","last_unstale":"2023-10-06T14:26:43.874971+0000","last_undegraded":"2023-10-06T14:26:43.874971+0000","last_fullsized":"2023-10-06T14:26:43.874971+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:43.866518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:43.866518+0000","last_clean_scrub_stamp":"2023-10-06T14:26:43.866518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:08.741049+0000","scrub_duration":0.053312574000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":924,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:47.113658+0000","last_change":"2023-10-06T14:26:47.115292+0000","last_active":"2023-10-06T14:26:47.113658+0000","last_peered":"2023-10-06T14:26:47.113658+0000","last_clean":"2023-10-06T14:26:47.113658+0000","last_became_active":"2023-10-06T13:54:19.531077+0000","last_became_peered":"2023-10-06T13:54:19.531077+0000","last_unstale":"2023-10-06T14:26:47.113658+0000","last_undegraded":"2023-10-06T14:26:47.113658+0000","last_fullsized":"2023-10-06T14:26:47.113658+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:47.112914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:47.112914+0000","last_clean_scrub_stamp":"2023-10-06T14:26:47.112914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:19.752892+0000","scrub_duration":0.045651023999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":934,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:40.376656+0000","last_change":"2023-10-06T14:26:40.378249+0000","last_active":"2023-10-06T14:26:40.376656+0000","last_peered":"2023-10-06T14:26:40.376656+0000","last_clean":"2023-10-06T14:26:40.376656+0000","last_became_active":"2023-10-06T14:06:28.630618+0000","last_became_peered":"2023-10-06T14:06:28.630618+0000","last_unstale":"2023-10-06T14:26:40.376656+0000","last_undegraded":"2023-10-06T14:26:40.376656+0000","last_fullsized":"2023-10-06T14:26:40.376656+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:40.375936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:40.375936+0000","last_clean_scrub_stamp":"2023-10-06T14:26:40.375936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:38:22.194622+0000","scrub_duration":0.067338567000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":943,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:35.321577+0000","last_change":"2023-10-06T14:26:35.327110+0000","last_active":"2023-10-06T14:26:35.321577+0000","last_peered":"2023-10-06T14:26:35.321577+0000","last_clean":"2023-10-06T14:26:35.321577+0000","last_became_active":"2023-10-06T13:54:21.813915+0000","last_became_peered":"2023-10-06T13:54:21.813915+0000","last_unstale":"2023-10-06T14:26:35.321577+0000","last_undegraded":"2023-10-06T14:26:35.321577+0000","last_fullsized":"2023-10-06T14:26:35.321577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:35.320741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:35.320741+0000","last_clean_scrub_stamp":"2023-10-06T14:26:35.320741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:23.967871+0000","scrub_duration":0.060312161000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":926,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:32.083574+0000","last_change":"2023-10-06T14:22:39.679043+0000","last_active":"2023-10-06T14:26:32.083574+0000","last_peered":"2023-10-06T14:26:32.083574+0000","last_clean":"2023-10-06T14:26:32.083574+0000","last_became_active":"2023-10-06T14:22:39.673285+0000","last_became_peered":"2023-10-06T14:22:39.673285+0000","last_unstale":"2023-10-06T14:26:32.083574+0000","last_undegraded":"2023-10-06T14:26:32.083574+0000","last_fullsized":"2023-10-06T14:26:32.083574+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":928,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:36.928323+0000","last_change":"2023-10-06T14:26:36.929962+0000","last_active":"2023-10-06T14:26:36.928323+0000","last_peered":"2023-10-06T14:26:36.928323+0000","last_clean":"2023-10-06T14:26:36.928323+0000","last_became_active":"2023-10-06T14:16:33.392050+0000","last_became_peered":"2023-10-06T14:16:33.392050+0000","last_unstale":"2023-10-06T14:26:36.928323+0000","last_undegraded":"2023-10-06T14:26:36.928323+0000","last_fullsized":"2023-10-06T14:26:36.928323+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:36.927606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:36.927606+0000","last_clean_scrub_stamp":"2023-10-06T14:26:36.927606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:29.494165+0000","scrub_duration":0.034420503999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":938,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:37.255284+0000","last_change":"2023-10-06T14:26:37.255284+0000","last_active":"2023-10-06T14:26:37.255284+0000","last_peered":"2023-10-06T14:26:37.255284+0000","last_clean":"2023-10-06T14:26:37.255284+0000","last_became_active":"2023-10-06T14:05:13.248166+0000","last_became_peered":"2023-10-06T14:05:13.248166+0000","last_unstale":"2023-10-06T14:26:37.255284+0000","last_undegraded":"2023-10-06T14:26:37.255284+0000","last_fullsized":"2023-10-06T14:26:37.255284+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:37.252750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:37.252750+0000","last_clean_scrub_stamp":"2023-10-06T14:26:37.252750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:56:31.690711+0000","scrub_duration":0.032023376999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":921,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:46.151441+0000","last_change":"2023-10-06T14:26:46.155677+0000","last_active":"2023-10-06T14:26:46.151441+0000","last_peered":"2023-10-06T14:26:46.151441+0000","last_clean":"2023-10-06T14:26:46.151441+0000","last_became_active":"2023-10-06T13:54:26.218851+0000","last_became_peered":"2023-10-06T13:54:26.218851+0000","last_unstale":"2023-10-06T14:26:46.151441+0000","last_undegraded":"2023-10-06T14:26:46.151441+0000","last_fullsized":"2023-10-06T14:26:46.151441+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:46.150724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:46.150724+0000","last_clean_scrub_stamp":"2023-10-06T14:26:46.150724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:05:58.492416+0000","scrub_duration":0.072809143000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":942,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.381372+0000","last_change":"2023-10-06T14:22:39.234795+0000","last_active":"2023-10-06T14:26:28.381372+0000","last_peered":"2023-10-06T14:26:28.381372+0000","last_clean":"2023-10-06T14:26:28.381372+0000","last_became_active":"2023-10-06T14:22:39.109893+0000","last_became_peered":"2023-10-06T14:22:39.109893+0000","last_unstale":"2023-10-06T14:26:28.381372+0000","last_undegraded":"2023-10-06T14:26:28.381372+0000","last_fullsized":"2023-10-06T14:26:28.381372+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":940,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:46.934884+0000","last_change":"2023-10-06T14:26:46.939646+0000","last_active":"2023-10-06T14:26:46.934884+0000","last_peered":"2023-10-06T14:26:46.934884+0000","last_clean":"2023-10-06T14:26:46.934884+0000","last_became_active":"2023-10-06T14:16:33.551586+0000","last_became_peered":"2023-10-06T14:16:33.551586+0000","last_unstale":"2023-10-06T14:26:46.934884+0000","last_undegraded":"2023-10-06T14:26:46.934884+0000","last_fullsized":"2023-10-06T14:26:46.934884+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:46.934161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:46.934161+0000","last_clean_scrub_stamp":"2023-10-06T14:26:46.934161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:55:04.465763+0000","scrub_duration":0.048811647999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":933,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:34.939197+0000","last_change":"2023-10-06T14:26:34.944308+0000","last_active":"2023-10-06T14:26:34.939197+0000","last_peered":"2023-10-06T14:26:34.939197+0000","last_clean":"2023-10-06T14:26:34.939197+0000","last_became_active":"2023-10-06T14:16:33.398349+0000","last_became_peered":"2023-10-06T14:16:33.398349+0000","last_unstale":"2023-10-06T14:26:34.939197+0000","last_undegraded":"2023-10-06T14:26:34.939197+0000","last_fullsized":"2023-10-06T14:26:34.939197+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:34.938407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:34.938407+0000","last_clean_scrub_stamp":"2023-10-06T14:26:34.938407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:48:11.028866+0000","scrub_duration":0.031095200999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":942,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:36.495949+0000","last_change":"2023-10-06T14:26:36.500436+0000","last_active":"2023-10-06T14:26:36.495949+0000","last_peered":"2023-10-06T14:26:36.495949+0000","last_clean":"2023-10-06T14:26:36.495949+0000","last_became_active":"2023-10-06T13:54:31.908571+0000","last_became_peered":"2023-10-06T13:54:31.908571+0000","last_unstale":"2023-10-06T14:26:36.495949+0000","last_undegraded":"2023-10-06T14:26:36.495949+0000","last_fullsized":"2023-10-06T14:26:36.495949+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:36.495123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:36.495123+0000","last_clean_scrub_stamp":"2023-10-06T14:26:36.495123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:21:09.895426+0000","scrub_duration":0.061798104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":932,"reported_epoch":970,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T14:26:32.075893+0000","last_change":"2023-10-06T14:26:36.482503+0000","last_active":"2023-10-06T14:26:32.075893+0000","last_peered":"2023-10-06T14:26:32.075893+0000","last_clean":"2023-10-06T14:26:32.075893+0000","last_became_active":"2023-10-06T14:22:39.078831+0000","last_became_peered":"2023-10-06T14:22:39.078831+0000","last_unstale":"2023-10-06T14:26:32.075893+0000","last_undegraded":"2023-10-06T14:26:32.075893+0000","last_fullsized":"2023-10-06T14:26:32.075893+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":933,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:39.351473+0000","last_change":"2023-10-06T14:26:39.353096+0000","last_active":"2023-10-06T14:26:39.351473+0000","last_peered":"2023-10-06T14:26:39.351473+0000","last_clean":"2023-10-06T14:26:39.351473+0000","last_became_active":"2023-10-06T14:05:13.230098+0000","last_became_peered":"2023-10-06T14:05:13.230098+0000","last_unstale":"2023-10-06T14:26:39.351473+0000","last_undegraded":"2023-10-06T14:26:39.351473+0000","last_fullsized":"2023-10-06T14:26:39.351473+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:39.350763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:39.350763+0000","last_clean_scrub_stamp":"2023-10-06T14:26:39.350763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:19:54.414820+0000","scrub_duration":0.064104507000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":942,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:39.196099+0000","last_change":"2023-10-06T14:26:39.200691+0000","last_active":"2023-10-06T14:26:39.196099+0000","last_peered":"2023-10-06T14:26:39.196099+0000","last_clean":"2023-10-06T14:26:39.196099+0000","last_became_active":"2023-10-06T14:16:33.635681+0000","last_became_peered":"2023-10-06T14:16:33.635681+0000","last_unstale":"2023-10-06T14:26:39.196099+0000","last_undegraded":"2023-10-06T14:26:39.196099+0000","last_fullsized":"2023-10-06T14:26:39.196099+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:39.195361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:39.195361+0000","last_clean_scrub_stamp":"2023-10-06T14:26:39.195361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:03:57.492939+0000","scrub_duration":0.040464247000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":942,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:41.020728+0000","last_change":"2023-10-06T14:26:41.022362+0000","last_active":"2023-10-06T14:26:41.020728+0000","last_peered":"2023-10-06T14:26:41.020728+0000","last_clean":"2023-10-06T14:26:41.020728+0000","last_became_active":"2023-10-06T14:16:33.269648+0000","last_became_peered":"2023-10-06T14:16:33.269648+0000","last_unstale":"2023-10-06T14:26:41.020728+0000","last_undegraded":"2023-10-06T14:26:41.020728+0000","last_fullsized":"2023-10-06T14:26:41.020728+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:41.020007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:41.020007+0000","last_clean_scrub_stamp":"2023-10-06T14:26:41.020007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:31.705678+0000","scrub_duration":0.032194962000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":943,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:34.467517+0000","last_change":"2023-10-06T14:26:38.444302+0000","last_active":"2023-10-06T14:26:34.467517+0000","last_peered":"2023-10-06T14:26:34.467517+0000","last_clean":"2023-10-06T14:26:34.467517+0000","last_became_active":"2023-10-06T14:22:39.121176+0000","last_became_peered":"2023-10-06T14:22:39.121176+0000","last_unstale":"2023-10-06T14:26:34.467517+0000","last_undegraded":"2023-10-06T14:26:34.467517+0000","last_fullsized":"2023-10-06T14:26:34.467517+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":936,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:34.523930+0000","last_change":"2023-10-06T14:26:34.528621+0000","last_active":"2023-10-06T14:26:34.523930+0000","last_peered":"2023-10-06T14:26:34.523930+0000","last_clean":"2023-10-06T14:26:34.523930+0000","last_became_active":"2023-10-06T13:54:27.642873+0000","last_became_peered":"2023-10-06T13:54:27.642873+0000","last_unstale":"2023-10-06T14:26:34.523930+0000","last_undegraded":"2023-10-06T14:26:34.523930+0000","last_fullsized":"2023-10-06T14:26:34.523930+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:34.522838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:34.522838+0000","last_clean_scrub_stamp":"2023-10-06T14:26:34.522838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:59:08.227916+0000","scrub_duration":0.081521365999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":929,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:28.371814+0000","last_change":"2023-10-06T14:22:39.756236+0000","last_active":"2023-10-06T14:26:28.371814+0000","last_peered":"2023-10-06T14:26:28.371814+0000","last_clean":"2023-10-06T14:26:28.371814+0000","last_became_active":"2023-10-06T14:22:39.733473+0000","last_became_peered":"2023-10-06T14:22:39.733473+0000","last_unstale":"2023-10-06T14:26:28.371814+0000","last_undegraded":"2023-10-06T14:26:28.371814+0000","last_fullsized":"2023-10-06T14:26:28.371814+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T13:53:14.910188+0000","last_clean_scrub_stamp":"2023-10-06T13:53:14.910188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"970'78","reported_seq":1070,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:34.515865+0000","last_change":"2023-10-06T14:26:34.521747+0000","last_active":"2023-10-06T14:26:34.515865+0000","last_peered":"2023-10-06T14:26:34.515865+0000","last_clean":"2023-10-06T14:26:34.515865+0000","last_became_active":"2023-10-06T14:16:33.632776+0000","last_became_peered":"2023-10-06T14:16:33.632776+0000","last_unstale":"2023-10-06T14:26:34.515865+0000","last_undegraded":"2023-10-06T14:26:34.515865+0000","last_fullsized":"2023-10-06T14:26:34.515865+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"970'78","last_scrub_stamp":"2023-10-06T14:26:34.514526+0000","last_deep_scrub":"970'78","last_deep_scrub_stamp":"2023-10-06T14:26:34.514526+0000","last_clean_scrub_stamp":"2023-10-06T14:26:34.514526+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:00.523848+0000","scrub_duration":0.114787636,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":922,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:42.132958+0000","last_change":"2023-10-06T14:26:42.134644+0000","last_active":"2023-10-06T14:26:42.132958+0000","last_peered":"2023-10-06T14:26:42.132958+0000","last_clean":"2023-10-06T14:26:42.132958+0000","last_became_active":"2023-10-06T13:54:26.193354+0000","last_became_peered":"2023-10-06T13:54:26.193354+0000","last_unstale":"2023-10-06T14:26:42.132958+0000","last_undegraded":"2023-10-06T14:26:42.132958+0000","last_fullsized":"2023-10-06T14:26:42.132958+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:42.132239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:42.132239+0000","last_clean_scrub_stamp":"2023-10-06T14:26:42.132239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:55:22.212657+0000","scrub_duration":0.056253063999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":944,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:33.365482+0000","last_change":"2023-10-06T14:26:33.367228+0000","last_active":"2023-10-06T14:26:33.365482+0000","last_peered":"2023-10-06T14:26:33.365482+0000","last_clean":"2023-10-06T14:26:33.365482+0000","last_became_active":"2023-10-06T13:54:22.005243+0000","last_became_peered":"2023-10-06T13:54:22.005243+0000","last_unstale":"2023-10-06T14:26:33.365482+0000","last_undegraded":"2023-10-06T14:26:33.365482+0000","last_fullsized":"2023-10-06T14:26:33.365482+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:33.364656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:33.364656+0000","last_clean_scrub_stamp":"2023-10-06T14:26:33.364656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:23:09.595356+0000","scrub_duration":0.043849388000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"970'8","reported_seq":952,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:37.931903+0000","last_change":"2023-10-06T14:26:37.936968+0000","last_active":"2023-10-06T14:26:37.931903+0000","last_peered":"2023-10-06T14:26:37.931903+0000","last_clean":"2023-10-06T14:26:37.931903+0000","last_became_active":"2023-10-06T14:05:13.458219+0000","last_became_peered":"2023-10-06T14:05:13.458219+0000","last_unstale":"2023-10-06T14:26:37.931903+0000","last_undegraded":"2023-10-06T14:26:37.931903+0000","last_fullsized":"2023-10-06T14:26:37.931903+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"970'8","last_scrub_stamp":"2023-10-06T14:26:37.931101+0000","last_deep_scrub":"970'8","last_deep_scrub_stamp":"2023-10-06T14:26:37.931101+0000","last_clean_scrub_stamp":"2023-10-06T14:26:37.931101+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:29.815005+0000","scrub_duration":0.09831413,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":944,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:43.006334+0000","last_change":"2023-10-06T14:26:43.007863+0000","last_active":"2023-10-06T14:26:43.006334+0000","last_peered":"2023-10-06T14:26:43.006334+0000","last_clean":"2023-10-06T14:26:43.006334+0000","last_became_active":"2023-10-06T14:22:39.269435+0000","last_became_peered":"2023-10-06T14:22:39.269435+0000","last_unstale":"2023-10-06T14:26:43.006334+0000","last_undegraded":"2023-10-06T14:26:43.006334+0000","last_fullsized":"2023-10-06T14:26:43.006334+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:43.005623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:43.005623+0000","last_clean_scrub_stamp":"2023-10-06T14:26:43.005623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:06:44.934607+0000","scrub_duration":0.033589740999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":934,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:45.112146+0000","last_change":"2023-10-06T14:26:45.113928+0000","last_active":"2023-10-06T14:26:45.112146+0000","last_peered":"2023-10-06T14:26:45.112146+0000","last_clean":"2023-10-06T14:26:45.112146+0000","last_became_active":"2023-10-06T14:16:33.650127+0000","last_became_peered":"2023-10-06T14:16:33.650127+0000","last_unstale":"2023-10-06T14:26:45.112146+0000","last_undegraded":"2023-10-06T14:26:45.112146+0000","last_fullsized":"2023-10-06T14:26:45.112146+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:45.111370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:45.111370+0000","last_clean_scrub_stamp":"2023-10-06T14:26:45.111370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:38.367962+0000","scrub_duration":0.062333412999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":924,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:46.863778+0000","last_change":"2023-10-06T14:26:46.863778+0000","last_active":"2023-10-06T14:26:46.863778+0000","last_peered":"2023-10-06T14:26:46.863778+0000","last_clean":"2023-10-06T14:26:46.863778+0000","last_became_active":"2023-10-06T13:54:22.160421+0000","last_became_peered":"2023-10-06T13:54:22.160421+0000","last_unstale":"2023-10-06T14:26:46.863778+0000","last_undegraded":"2023-10-06T14:26:46.863778+0000","last_fullsized":"2023-10-06T14:26:46.863778+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:46.861353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:46.861353+0000","last_clean_scrub_stamp":"2023-10-06T14:26:46.861353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:52:25.904881+0000","scrub_duration":0.052787556999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":943,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:35.440991+0000","last_change":"2023-10-06T14:26:35.442788+0000","last_active":"2023-10-06T14:26:35.440991+0000","last_peered":"2023-10-06T14:26:35.440991+0000","last_clean":"2023-10-06T14:26:35.440991+0000","last_became_active":"2023-10-06T13:54:25.349894+0000","last_became_peered":"2023-10-06T13:54:25.349894+0000","last_unstale":"2023-10-06T14:26:35.440991+0000","last_undegraded":"2023-10-06T14:26:35.440991+0000","last_fullsized":"2023-10-06T14:26:35.440991+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:35.440184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:35.440184+0000","last_clean_scrub_stamp":"2023-10-06T14:26:35.440184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:26.848807+0000","scrub_duration":0.052068601999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":957,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:34.317404+0000","last_change":"2023-10-06T14:26:34.319556+0000","last_active":"2023-10-06T14:26:34.317404+0000","last_peered":"2023-10-06T14:26:34.317404+0000","last_clean":"2023-10-06T14:26:34.317404+0000","last_became_active":"2023-10-06T14:22:39.979167+0000","last_became_peered":"2023-10-06T14:22:39.979167+0000","last_unstale":"2023-10-06T14:26:34.317404+0000","last_undegraded":"2023-10-06T14:26:34.317404+0000","last_fullsized":"2023-10-06T14:26:34.317404+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":836,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:34.316567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:34.316567+0000","last_clean_scrub_stamp":"2023-10-06T14:26:34.316567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:33:39.123933+0000","scrub_duration":0.035959370999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":954,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:31.449380+0000","last_change":"2023-10-06T14:26:31.449380+0000","last_active":"2023-10-06T14:26:31.449380+0000","last_peered":"2023-10-06T14:26:31.449380+0000","last_clean":"2023-10-06T14:26:31.449380+0000","last_became_active":"2023-10-06T14:05:13.191757+0000","last_became_peered":"2023-10-06T14:05:13.191757+0000","last_unstale":"2023-10-06T14:26:31.449380+0000","last_undegraded":"2023-10-06T14:26:31.449380+0000","last_fullsized":"2023-10-06T14:26:31.449380+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:31.445079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:31.445079+0000","last_clean_scrub_stamp":"2023-10-06T14:26:31.445079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:17:11.350420+0000","scrub_duration":0.083960115000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":923,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:36.882247+0000","last_change":"2023-10-06T14:26:36.882247+0000","last_active":"2023-10-06T14:26:36.882247+0000","last_peered":"2023-10-06T14:26:36.882247+0000","last_clean":"2023-10-06T14:26:36.882247+0000","last_became_active":"2023-10-06T13:54:22.144576+0000","last_became_peered":"2023-10-06T13:54:22.144576+0000","last_unstale":"2023-10-06T14:26:36.882247+0000","last_undegraded":"2023-10-06T14:26:36.882247+0000","last_fullsized":"2023-10-06T14:26:36.882247+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:36.878675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:36.878675+0000","last_clean_scrub_stamp":"2023-10-06T14:26:36.878675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:18:58.847490+0000","scrub_duration":0.050518194000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":941,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:46.697951+0000","last_change":"2023-10-06T14:26:46.699751+0000","last_active":"2023-10-06T14:26:46.697951+0000","last_peered":"2023-10-06T14:26:46.697951+0000","last_clean":"2023-10-06T14:26:46.697951+0000","last_became_active":"2023-10-06T13:54:19.734381+0000","last_became_peered":"2023-10-06T13:54:19.734381+0000","last_unstale":"2023-10-06T14:26:46.697951+0000","last_undegraded":"2023-10-06T14:26:46.697951+0000","last_fullsized":"2023-10-06T14:26:46.697951+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:46.697067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:46.697067+0000","last_clean_scrub_stamp":"2023-10-06T14:26:46.697067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:16:20.063672+0000","scrub_duration":0.065551264999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":948,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:30.418675+0000","last_change":"2023-10-06T14:26:30.426357+0000","last_active":"2023-10-06T14:26:30.418675+0000","last_peered":"2023-10-06T14:26:30.418675+0000","last_clean":"2023-10-06T14:26:30.418675+0000","last_became_active":"2023-10-06T14:05:13.442612+0000","last_became_peered":"2023-10-06T14:05:13.442612+0000","last_unstale":"2023-10-06T14:26:30.418675+0000","last_undegraded":"2023-10-06T14:26:30.418675+0000","last_fullsized":"2023-10-06T14:26:30.418675+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:30.417803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:30.417803+0000","last_clean_scrub_stamp":"2023-10-06T14:26:30.417803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:23:39.298614+0000","scrub_duration":0.036831065000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719477128,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59132,"kb_used_data":5940,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312708,"statfs":{"total":96636764160,"available":96576212992,"internally_reserved":0,"allocated":6082560,"data_stored":3417737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":11.946,"5min":12.488,"15min":13.279},"min":{"1min":6.3920000000000003,"5min":4.6070000000000002,"15min":4.6070000000000002},"max":{"1min":52.079999999999998,"5min":91.744,"15min":109.691},"last":11.362},{"interface":"front","average":{"1min":12.875999999999999,"5min":12.664,"15min":14.576000000000001},"min":{"1min":4.1870000000000003,"5min":4.1870000000000003,"15min":4.1870000000000003},"max":{"1min":44.232999999999997,"5min":67.936000000000007,"15min":166.33699999999999},"last":10.596}]},{"osd":2,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":12.112,"5min":11.595000000000001,"15min":13.598000000000001},"min":{"1min":7.9850000000000003,"5min":6.3920000000000003,"15min":4.9800000000000004},"max":{"1min":18.631,"5min":64.661000000000001,"15min":155.68799999999999},"last":20.135999999999999},{"interface":"front","average":{"1min":15.757999999999999,"5min":13.82,"15min":16.335000000000001},"min":{"1min":9.0660000000000007,"5min":7.3049999999999997,"15min":6.6059999999999999},"max":{"1min":47.368000000000002,"5min":92.335999999999999,"15min":212.16399999999999},"last":21.236000000000001}]},{"osd":3,"last update":"Fri Oct 6 14:26:49 2023","interfaces":[{"interface":"back","average":{"1min":12.4,"5min":25.032,"15min":32.039999999999999},"min":{"1min":8.2949999999999999,"5min":8.2949999999999999,"15min":8.2949999999999999},"max":{"1min":19.728999999999999,"5min":310.19,"15min":310.19},"last":17.805},{"interface":"front","average":{"1min":12.678000000000001,"5min":17.033999999999999,"15min":18.442},"min":{"1min":7.5030000000000001,"5min":7.5030000000000001,"15min":7.5030000000000001},"max":{"1min":18.052,"5min":65.375,"15min":65.375},"last":18.382999999999999}]},{"osd":4,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":15290.522000000001,"5min":3070.721,"15min":1038.3879999999999},"min":{"1min":9.2720000000000002,"5min":7.8520000000000003,"15min":7.0190000000000001},"max":{"1min":39693.332999999999,"5min":39693.332999999999,"15min":39693.332999999999},"last":19.475000000000001},{"interface":"front","average":{"1min":15323.151,"5min":3078.1880000000001,"15min":1039.8889999999999},"min":{"1min":14.579000000000001,"5min":8.5679999999999996,"15min":7.3239999999999998},"max":{"1min":39697.402000000002,"5min":39697.402000000002,"15min":39697.402000000002},"last":13.252000000000001}]},{"osd":5,"last update":"Fri Oct 6 14:26:49 2023","interfaces":[{"interface":"back","average":{"1min":14.340999999999999,"5min":16.838000000000001,"15min":90.614999999999995},"min":{"1min":9.3989999999999991,"5min":9.1199999999999992,"15min":9.1199999999999992},"max":{"1min":19.346,"5min":69.561999999999998,"15min":8890.4069999999992},"last":14.778},{"interface":"front","average":{"1min":12.846,"5min":16.812000000000001,"15min":90.933999999999997},"min":{"1min":7.3239999999999998,"5min":7.3239999999999998,"15min":7.3239999999999998},"max":{"1min":21.536000000000001,"5min":139.78800000000001,"15min":8888.7939999999999},"last":15.483000000000001}]},{"osd":6,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.563000000000001,"5min":15.667999999999999,"15min":1233.97},"min":{"1min":9.9359999999999999,"5min":8.4710000000000001,"15min":7.3529999999999998},"max":{"1min":20.318000000000001,"5min":78.221999999999994,"15min":47304.226999999999},"last":21.82},{"interface":"front","average":{"1min":17.091000000000001,"5min":18.989000000000001,"15min":1237.328},"min":{"1min":11.202999999999999,"5min":10.666,"15min":8.9949999999999992},"max":{"1min":24.579000000000001,"5min":116.84999999999999,"15min":47302.955999999998},"last":23.638000000000002}]},{"osd":7,"last update":"Fri Oct 6 14:26:33 2023","interfaces":[{"interface":"back","average":{"1min":17.274999999999999,"5min":20.236000000000001,"15min":22.026},"min":{"1min":11.840999999999999,"5min":10.992000000000001,"15min":9.0250000000000004},"max":{"1min":23.812000000000001,"5min":78.769000000000005,"15min":172.16300000000001},"last":22.402000000000001},{"interface":"front","average":{"1min":15.042999999999999,"5min":20.027999999999999,"15min":21.492000000000001},"min":{"1min":9.7680000000000007,"5min":9.7680000000000007,"15min":9.7560000000000002},"max":{"1min":30.442,"5min":134.17400000000001,"15min":180.739},"last":16.116}]}]},{"osd":0,"up_from":16,"seq":68719477144,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58740,"kb_used_data":5548,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313100,"statfs":{"total":96636764160,"available":96576614400,"internally_reserved":0,"allocated":5681152,"data_stored":3019897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 14:26:15 2023","interfaces":[{"interface":"back","average":{"1min":19.928999999999998,"5min":17.440999999999999,"15min":16.654},"min":{"1min":6.7800000000000002,"5min":5.4980000000000002,"15min":4.9720000000000004},"max":{"1min":97.855999999999995,"5min":135.13800000000001,"15min":342.61399999999998},"last":8.5600000000000005},{"interface":"front","average":{"1min":21.222000000000001,"5min":19.111999999999998,"15min":17.628},"min":{"1min":5.9409999999999998,"5min":5.8810000000000002,"15min":5.5960000000000001},"max":{"1min":144.221,"5min":144.221,"15min":337.48000000000002},"last":11.778}]},{"osd":2,"last update":"Fri Oct 6 14:26:25 2023","interfaces":[{"interface":"back","average":{"1min":24.416,"5min":21.916,"15min":19.238},"min":{"1min":5.5030000000000001,"5min":5.5030000000000001,"15min":5.5030000000000001},"max":{"1min":231.68000000000001,"5min":231.68000000000001,"15min":231.68000000000001},"last":8.8450000000000006},{"interface":"front","average":{"1min":20.329000000000001,"5min":17.545000000000002,"15min":17.381},"min":{"1min":6.6470000000000002,"5min":5.718,"15min":5.718},"max":{"1min":121.98399999999999,"5min":181.06700000000001,"15min":181.06700000000001},"last":9.4510000000000005}]},{"osd":3,"last update":"Fri Oct 6 14:26:46 2023","interfaces":[{"interface":"back","average":{"1min":14.472,"5min":22.420999999999999,"15min":24.835000000000001},"min":{"1min":9.1199999999999992,"5min":7.4139999999999997,"15min":7.4139999999999997},"max":{"1min":38.506999999999998,"5min":189.10499999999999,"15min":189.10499999999999},"last":11.186},{"interface":"front","average":{"1min":18.135000000000002,"5min":23.192,"15min":24.527000000000001},"min":{"1min":11.356999999999999,"5min":6.9850000000000003,"15min":6.9850000000000003},"max":{"1min":46.523000000000003,"5min":163.60900000000001,"15min":163.60900000000001},"last":15.698}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":3655.643,"5min":746.56799999999998,"15min":262.995},"min":{"1min":13.273999999999999,"5min":8.7420000000000009,"15min":6.9269999999999996},"max":{"1min":39913.453999999998,"5min":39913.453999999998,"15min":39913.453999999998},"last":17.983000000000001},{"interface":"front","average":{"1min":3644.9029999999998,"5min":742.30499999999995,"15min":261.23700000000002},"min":{"1min":7.2050000000000001,"5min":7.2050000000000001,"15min":7.2050000000000001},"max":{"1min":39912.421999999999,"5min":39912.421999999999,"15min":39912.421999999999},"last":10.377000000000001}]},{"osd":5,"last update":"Fri Oct 6 14:26:50 2023","interfaces":[{"interface":"back","average":{"1min":25.170999999999999,"5min":18.625,"15min":70.325999999999993},"min":{"1min":11.279,"5min":7.5209999999999999,"15min":7.5209999999999999},"max":{"1min":172.108,"5min":172.108,"15min":6380.991},"last":15.058999999999999},{"interface":"front","average":{"1min":19.899999999999999,"5min":21.440000000000001,"15min":65.507000000000005},"min":{"1min":8.5370000000000008,"5min":8.5370000000000008,"15min":8.5370000000000008},"max":{"1min":87.198999999999998,"5min":145.59299999999999,"15min":6266.0159999999996},"last":21.154}]},{"osd":6,"last update":"Fri Oct 6 14:26:05 2023","interfaces":[{"interface":"back","average":{"1min":13.15,"5min":20.032,"15min":725.85500000000002},"min":{"1min":7.7789999999999999,"5min":7.1479999999999997,"15min":7.1479999999999997},"max":{"1min":26.087,"5min":177.73699999999999,"15min":46416.196000000004},"last":13.755000000000001},{"interface":"front","average":{"1min":15.784000000000001,"5min":22.670000000000002,"15min":724.05899999999997},"min":{"1min":10.669,"5min":8.6609999999999996,"15min":6.9459999999999997},"max":{"1min":21.206,"5min":181.625,"15min":46403.283000000003},"last":18.696000000000002}]},{"osd":7,"last update":"Fri Oct 6 14:26:33 2023","interfaces":[{"interface":"back","average":{"1min":23.925000000000001,"5min":24.867999999999999,"15min":1322.9580000000001},"min":{"1min":10.121,"5min":10.121,"15min":9.8209999999999997},"max":{"1min":96.147000000000006,"5min":178.86500000000001,"15min":67028.107999999993},"last":21.806999999999999},{"interface":"front","average":{"1min":19.460000000000001,"5min":20.870999999999999,"15min":1320.289},"min":{"1min":7.8730000000000002,"5min":7.8730000000000002,"15min":7.8730000000000002},"max":{"1min":89.558999999999997,"5min":178.38900000000001,"15min":67001.737999999998},"last":17.111000000000001}]}]},{"osd":4,"up_from":430,"seq":1846835937660,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58744,"kb_used_data":5552,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313096,"statfs":{"total":96636764160,"available":96576610304,"internally_reserved":0,"allocated":5685248,"data_stored":3019902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":61.039999999999999,"5min":21.318999999999999,"15min":15.34},"min":{"1min":7.0979999999999999,"5min":4.508,"15min":4.085},"max":{"1min":469.096,"5min":469.096,"15min":469.096},"last":7.8730000000000002},{"interface":"front","average":{"1min":60.287999999999997,"5min":20.288,"15min":17.068999999999999},"min":{"1min":8.5890000000000004,"5min":4.758,"15min":4.5999999999999996},"max":{"1min":423.42000000000002,"5min":423.42000000000002,"15min":423.42000000000002},"last":11.845000000000001}]},{"osd":1,"last update":"Fri Oct 6 14:26:13 2023","interfaces":[{"interface":"back","average":{"1min":81.888000000000005,"5min":27.257000000000001,"15min":19.550000000000001},"min":{"1min":7.3289999999999997,"5min":5.9850000000000003,"15min":4.9390000000000001},"max":{"1min":469.72800000000001,"5min":469.72800000000001,"15min":469.72800000000001},"last":11.207000000000001},{"interface":"front","average":{"1min":77.108999999999995,"5min":26.469999999999999,"15min":21.765999999999998},"min":{"1min":5.9320000000000004,"5min":5.9320000000000004,"15min":5.9320000000000004},"max":{"1min":425.584,"5min":425.584,"15min":425.584},"last":7.048}]},{"osd":2,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":113.371,"5min":34.213000000000001,"15min":23.704999999999998},"min":{"1min":7.6429999999999998,"5min":6.8159999999999998,"15min":6.4770000000000003},"max":{"1min":530.30399999999997,"5min":530.30399999999997,"15min":530.30399999999997},"last":13.303000000000001},{"interface":"front","average":{"1min":117.30200000000001,"5min":37.220999999999997,"15min":26.550999999999998},"min":{"1min":10.667999999999999,"5min":7.0750000000000002,"15min":6.5780000000000003},"max":{"1min":530.86099999999999,"5min":530.86099999999999,"15min":530.86099999999999},"last":16.925999999999998}]},{"osd":3,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":58.695,"5min":25.254999999999999,"15min":19.239000000000001},"min":{"1min":11.391,"5min":9.9220000000000006,"15min":9.9220000000000006},"max":{"1min":470.80099999999999,"5min":470.80099999999999,"15min":470.80099999999999},"last":13.888999999999999},{"interface":"front","average":{"1min":49.429000000000002,"5min":22.123999999999999,"15min":17.120000000000001},"min":{"1min":6.7489999999999997,"5min":6.7489999999999997,"15min":6.7489999999999997},"max":{"1min":422.88,"5min":422.88,"15min":422.88},"last":10.025}]},{"osd":5,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":58.246000000000002,"5min":23.637,"15min":76.748000000000005},"min":{"1min":8.8409999999999993,"5min":8.2230000000000008,"15min":5.9509999999999996},"max":{"1min":431.57900000000001,"5min":431.57900000000001,"15min":8813.2510000000002},"last":8.5579999999999998},{"interface":"front","average":{"1min":62.423000000000002,"5min":22.936,"15min":77.950000000000003},"min":{"1min":6.5590000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"max":{"1min":537.68399999999997,"5min":537.68399999999997,"15min":8811.7579999999998},"last":14.606999999999999}]},{"osd":6,"last update":"Fri Oct 6 14:26:13 2023","interfaces":[{"interface":"back","average":{"1min":96.150999999999996,"5min":31.527000000000001,"15min":920.85000000000002},"min":{"1min":9.7840000000000007,"5min":6.9470000000000001,"15min":6.9470000000000001},"max":{"1min":563.85900000000004,"5min":563.85900000000004,"15min":44817.595000000001},"last":16.234000000000002},{"interface":"front","average":{"1min":84.043000000000006,"5min":30.527999999999999,"15min":922.12},"min":{"1min":12.715,"5min":7.5830000000000002,"15min":7.3789999999999996},"max":{"1min":471.80700000000002,"5min":471.80700000000002,"15min":44816.752999999997},"last":10.625999999999999}]},{"osd":7,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":50.648000000000003,"5min":23.149999999999999,"15min":1630.123},"min":{"1min":9.8119999999999994,"5min":9.7119999999999997,"15min":7.569},"max":{"1min":564.822,"5min":564.822,"15min":65382.432000000001},"last":17.658000000000001},{"interface":"front","average":{"1min":45.387999999999998,"5min":21.542999999999999,"15min":1627.549},"min":{"1min":13.016999999999999,"5min":7.8639999999999999,"15min":7.4059999999999997},"max":{"1min":433.30799999999999,"5min":433.30799999999999,"15min":65371.873},"last":12.403}]}]},{"osd":3,"up_from":834,"seq":3582002725223,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58740,"kb_used_data":5548,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313100,"statfs":{"total":96636764160,"available":96576614400,"internally_reserved":0,"allocated":5681152,"data_stored":3019897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:40 2023","interfaces":[{"interface":"back","average":{"1min":11.488,"5min":22.166,"15min":30.718},"min":{"1min":5.7679999999999998,"5min":4.8559999999999999,"15min":4.8559999999999999},"max":{"1min":46.276000000000003,"5min":188.56700000000001,"15min":188.56700000000001},"last":7.3540000000000001},{"interface":"front","average":{"1min":13.634,"5min":23.224,"15min":30.103000000000002},"min":{"1min":6.4980000000000002,"5min":5.827,"15min":5.827},"max":{"1min":49.347000000000001,"5min":214.209,"15min":214.209},"last":14.516999999999999}]},{"osd":1,"last update":"Fri Oct 6 14:26:40 2023","interfaces":[{"interface":"back","average":{"1min":20.433,"5min":34.404000000000003,"15min":33.829999999999998},"min":{"1min":7.7089999999999996,"5min":7.4409999999999998,"15min":7.4409999999999998},"max":{"1min":122.386,"5min":993.702,"15min":993.702},"last":11.894},{"interface":"front","average":{"1min":22.824000000000002,"5min":36.021000000000001,"15min":35.810000000000002},"min":{"1min":5.8869999999999996,"5min":5.8869999999999996,"15min":5.8869999999999996},"max":{"1min":147.75800000000001,"5min":991.50599999999997,"15min":991.50599999999997},"last":8.5050000000000008}]},{"osd":2,"last update":"Fri Oct 6 14:26:40 2023","interfaces":[{"interface":"back","average":{"1min":15.666,"5min":25.468,"15min":32.722999999999999},"min":{"1min":8.0190000000000001,"5min":8.0190000000000001,"15min":8.0190000000000001},"max":{"1min":59.93,"5min":230.25399999999999,"15min":230.25399999999999},"last":16.626999999999999},{"interface":"front","average":{"1min":15.298999999999999,"5min":23.792000000000002,"15min":29.678000000000001},"min":{"1min":8.3569999999999993,"5min":8.1959999999999997,"15min":8.1959999999999997},"max":{"1min":58.402000000000001,"5min":165.553,"15min":165.553},"last":11.282999999999999}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":2736.415,"5min":573.53200000000004,"15min":216.17099999999999},"min":{"1min":10.223000000000001,"5min":7.1020000000000003,"15min":7.1020000000000003},"max":{"1min":40828.069000000003,"5min":40828.069000000003,"15min":40828.069000000003},"last":16.015000000000001},{"interface":"front","average":{"1min":18.699999999999999,"5min":28.707999999999998,"15min":32.601999999999997},"min":{"1min":8.6150000000000002,"5min":8.6150000000000002,"15min":8.6150000000000002},"max":{"1min":100.607,"5min":194.14599999999999,"15min":194.14599999999999},"last":13.231999999999999}]},{"osd":5,"last update":"Fri Oct 6 14:26:40 2023","interfaces":[{"interface":"back","average":{"1min":18.367000000000001,"5min":26.66,"15min":33.622},"min":{"1min":9.7880000000000003,"5min":9.7880000000000003,"15min":9.7880000000000003},"max":{"1min":78.909000000000006,"5min":196.50200000000001,"15min":196.50200000000001},"last":13.795999999999999},{"interface":"front","average":{"1min":16.731000000000002,"5min":23.861000000000001,"15min":29.673999999999999},"min":{"1min":8.2129999999999992,"5min":7.9329999999999998,"15min":7.9329999999999998},"max":{"1min":60.256,"5min":177.739,"15min":177.739},"last":9.1809999999999992}]},{"osd":6,"last update":"Fri Oct 6 14:26:40 2023","interfaces":[{"interface":"back","average":{"1min":24.187999999999999,"5min":33.337000000000003,"15min":45.566000000000003},"min":{"1min":9.1679999999999993,"5min":9.1679999999999993,"15min":9.1679999999999993},"max":{"1min":127.839,"5min":381.13,"15min":381.13},"last":9.7509999999999994},{"interface":"front","average":{"1min":24.706,"5min":29.417999999999999,"15min":36.106999999999999},"min":{"1min":9.0510000000000002,"5min":9.0510000000000002,"15min":9.0510000000000002},"max":{"1min":123.29600000000001,"5min":245.16200000000001,"15min":245.16200000000001},"last":17.616}]},{"osd":7,"last update":"Fri Oct 6 14:26:40 2023","interfaces":[{"interface":"back","average":{"1min":22.215,"5min":25.847999999999999,"15min":28.803999999999998},"min":{"1min":9.8019999999999996,"5min":9.8019999999999996,"15min":9.8019999999999996},"max":{"1min":95.766999999999996,"5min":227.42599999999999,"15min":227.42599999999999},"last":18.454000000000001},{"interface":"front","average":{"1min":20.015999999999998,"5min":23.420999999999999,"15min":24.927},"min":{"1min":10.595000000000001,"5min":10.595000000000001,"15min":10.595000000000001},"max":{"1min":86.596999999999994,"5min":164.09899999999999,"15min":164.09899999999999},"last":12.662000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509837,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58740,"kb_used_data":5548,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313100,"statfs":{"total":96636764160,"available":96576614400,"internally_reserved":0,"allocated":5681152,"data_stored":3019897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":12.117000000000001,"5min":11.677,"15min":16.064},"min":{"1min":7.3819999999999997,"5min":5.452,"15min":5.3630000000000004},"max":{"1min":24.227,"5min":31.484000000000002,"15min":182.96600000000001},"last":6.6180000000000003},{"interface":"front","average":{"1min":10.268000000000001,"5min":12.111000000000001,"15min":17.263000000000002},"min":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"max":{"1min":17.821000000000002,"5min":44.423000000000002,"15min":226.68600000000001},"last":12.587999999999999}]},{"osd":1,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":18.187000000000001,"5min":16.309999999999999,"15min":17.614000000000001},"min":{"1min":7.6710000000000003,"5min":7.3630000000000004,"15min":6.0970000000000004},"max":{"1min":100.065,"5min":101.874,"15min":148.20400000000001},"last":22.146999999999998},{"interface":"front","average":{"1min":12.387,"5min":14.256,"15min":17.227},"min":{"1min":6.6840000000000002,"5min":6.4630000000000001,"15min":6.3140000000000001},"max":{"1min":53.959000000000003,"5min":161.97499999999999,"15min":229.82499999999999},"last":20.353999999999999}]},{"osd":3,"last update":"Fri Oct 6 14:26:50 2023","interfaces":[{"interface":"back","average":{"1min":18.111999999999998,"5min":17.122,"15min":16.521999999999998},"min":{"1min":8.1780000000000008,"5min":8.1780000000000008,"15min":8.1780000000000008},"max":{"1min":62.646999999999998,"5min":62.646999999999998,"15min":62.646999999999998},"last":16.082999999999998},{"interface":"front","average":{"1min":17.864000000000001,"5min":15.199,"15min":14.981},"min":{"1min":8.2080000000000002,"5min":8.2080000000000002,"15min":8.2080000000000002},"max":{"1min":64.054000000000002,"5min":64.054000000000002,"15min":64.054000000000002},"last":8.2080000000000002}]},{"osd":4,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":14883.07,"5min":2990.154,"15min":1010.401},"min":{"1min":6.8150000000000004,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":38405.120000000003,"5min":38405.120000000003,"15min":38405.120000000003},"last":17.326000000000001},{"interface":"front","average":{"1min":14785.589,"5min":2970.7600000000002,"15min":1006.212},"min":{"1min":7.8520000000000003,"5min":7.6550000000000002,"15min":7.6550000000000002},"max":{"1min":38264.116000000002,"5min":38264.116000000002,"15min":38264.116000000002},"last":8.9730000000000008}]},{"osd":5,"last update":"Fri Oct 6 14:25:51 2023","interfaces":[{"interface":"back","average":{"1min":16.574999999999999,"5min":16.280999999999999,"15min":92.066999999999993},"min":{"1min":8.2560000000000002,"5min":8.2560000000000002,"15min":7.6589999999999998},"max":{"1min":32.328000000000003,"5min":50.210000000000001,"15min":9721.0490000000009},"last":15.452},{"interface":"front","average":{"1min":14.137,"5min":16.710999999999999,"15min":96.823999999999998},"min":{"1min":8.6099999999999994,"5min":8.6099999999999994,"15min":8.6099999999999994},"max":{"1min":24.001000000000001,"5min":73.320999999999998,"15min":9793.7890000000007},"last":10.128}]},{"osd":6,"last update":"Fri Oct 6 14:26:07 2023","interfaces":[{"interface":"back","average":{"1min":14.999000000000001,"5min":17.789000000000001,"15min":1309.6320000000001},"min":{"1min":8.8689999999999998,"5min":8.8689999999999998,"15min":8.8689999999999998},"max":{"1min":23.431999999999999,"5min":44.606999999999999,"15min":44531.741000000002},"last":13.191000000000001},{"interface":"front","average":{"1min":16.434000000000001,"5min":16.972000000000001,"15min":1305.6900000000001},"min":{"1min":7.5330000000000004,"5min":7.141,"15min":7.141},"max":{"1min":49.576999999999998,"5min":49.576999999999998,"15min":44501.423000000003},"last":19.257999999999999}]},{"osd":7,"last update":"Fri Oct 6 14:26:36 2023","interfaces":[{"interface":"back","average":{"1min":20.045999999999999,"5min":20.690000000000001,"15min":1733.6900000000001},"min":{"1min":9.5600000000000005,"5min":9.5600000000000005,"15min":8.6189999999999998},"max":{"1min":73.903000000000006,"5min":73.903000000000006,"15min":69773.953999999998},"last":18.067},{"interface":"front","average":{"1min":20.099,"5min":19.152999999999999,"15min":1723.25},"min":{"1min":10.614000000000001,"5min":9.5909999999999993,"15min":8.5429999999999993},"max":{"1min":78.433999999999997,"5min":78.433999999999997,"15min":69547.547999999995},"last":21.373000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542536,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62840,"kb_used_data":5552,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94309000,"statfs":{"total":96636764160,"available":96572416000,"internally_reserved":0,"allocated":5685248,"data_stored":3019902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":8.6549999999999994,"5min":11.162000000000001,"15min":18.341000000000001},"min":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.4050000000000002},"max":{"1min":12.99,"5min":94.040999999999997,"15min":574.78800000000001},"last":13.611000000000001},{"interface":"front","average":{"1min":11.332000000000001,"5min":15.573,"15min":21.341999999999999},"min":{"1min":7.5309999999999997,"5min":7.5309999999999997,"15min":4.5140000000000002},"max":{"1min":17.783999999999999,"5min":130.036,"15min":522.72699999999998},"last":15.172000000000001}]},{"osd":1,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.231,"5min":27.030999999999999,"15min":24.16},"min":{"1min":7.3150000000000004,"5min":5.4059999999999997,"15min":5.4059999999999997},"max":{"1min":14.238,"5min":1271.6179999999999,"15min":1271.6179999999999},"last":11.528},{"interface":"front","average":{"1min":12.869,"5min":30.803000000000001,"15min":27.870999999999999},"min":{"1min":9.9900000000000002,"5min":6.8639999999999999,"15min":6.8639999999999999},"max":{"1min":17.446999999999999,"5min":1322.71,"15min":1322.71},"last":12.093999999999999}]},{"osd":2,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.842000000000001,"5min":14.388999999999999,"15min":19.920999999999999},"min":{"1min":9.4809999999999999,"5min":6.1660000000000004,"15min":5.532},"max":{"1min":30.937999999999999,"5min":101.197,"15min":525.81600000000003},"last":12.724},{"interface":"front","average":{"1min":11.125999999999999,"5min":17.469999999999999,"15min":24.055},"min":{"1min":6.7409999999999997,"5min":6.4509999999999996,"15min":6.4509999999999996},"max":{"1min":27.02,"5min":99.492999999999995,"15min":468.70400000000001},"last":8.6980000000000004}]},{"osd":3,"last update":"Fri Oct 6 14:26:43 2023","interfaces":[{"interface":"back","average":{"1min":17.905999999999999,"5min":19.57,"15min":19.963000000000001},"min":{"1min":9.3699999999999992,"5min":6.8470000000000004,"15min":6.8470000000000004},"max":{"1min":75.385999999999996,"5min":126.55500000000001,"15min":126.55500000000001},"last":16.968},{"interface":"front","average":{"1min":11.311,"5min":18.966999999999999,"15min":22.920000000000002},"min":{"1min":7.6619999999999999,"5min":6.0389999999999997,"15min":6.0389999999999997},"max":{"1min":16.341000000000001,"5min":236.27000000000001,"15min":236.27000000000001},"last":9.4619999999999997}]},{"osd":4,"last update":"Fri Oct 6 14:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15821.066000000001,"5min":3179.7730000000001,"15min":1076.7719999999999},"min":{"1min":10.141,"5min":9.1189999999999998,"15min":8.7490000000000006},"max":{"1min":38720.860999999997,"5min":38720.860999999997,"15min":38720.860999999997},"last":14.329000000000001},{"interface":"front","average":{"1min":15816.687,"5min":3177.5610000000001,"15min":1074.1969999999999},"min":{"1min":6.782,"5min":6.782,"15min":6.782},"max":{"1min":38719.173000000003,"5min":38719.173000000003,"15min":38719.173000000003},"last":15.890000000000001}]},{"osd":5,"last update":"Fri Oct 6 14:26:51 2023","interfaces":[{"interface":"back","average":{"1min":16.805,"5min":21.788,"15min":105.855},"min":{"1min":10.189,"5min":8.8800000000000008,"15min":8.8800000000000008},"max":{"1min":28.484000000000002,"5min":219.06200000000001,"15min":8032.4189999999999},"last":16.449000000000002},{"interface":"front","average":{"1min":17.248999999999999,"5min":21.498000000000001,"15min":108.09099999999999},"min":{"1min":11.912000000000001,"5min":7.1459999999999999,"15min":7.1459999999999999},"max":{"1min":29.532,"5min":171.041,"15min":8030.5749999999998},"last":18.827999999999999}]},{"osd":7,"last update":"Fri Oct 6 14:26:38 2023","interfaces":[{"interface":"back","average":{"1min":16.32,"5min":19.835000000000001,"15min":1130.444},"min":{"1min":13.643000000000001,"5min":11.01,"15min":11.01},"max":{"1min":19.600999999999999,"5min":93.355999999999995,"15min":69750.712},"last":18.084},{"interface":"front","average":{"1min":10.865,"5min":14.667,"15min":1124.432},"min":{"1min":6.806,"5min":6.4770000000000003,"15min":5.984},"max":{"1min":14.300000000000001,"5min":121.146,"15min":69749.794999999998},"last":10.634}]}]},{"osd":5,"up_from":659,"seq":2830383448445,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77180,"kb_used_data":5940,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94294660,"statfs":{"total":96636764160,"available":96557731840,"internally_reserved":0,"allocated":6082560,"data_stored":3417737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":10.645,"5min":13.339,"15min":38.990000000000002},"min":{"1min":4.6189999999999998,"5min":4.3339999999999996,"15min":4.1879999999999997},"max":{"1min":22.547999999999998,"5min":173.54599999999999,"15min":680.28200000000004},"last":11.206},{"interface":"front","average":{"1min":7.4779999999999998,"5min":11.548999999999999,"15min":29.652000000000001},"min":{"1min":4.7430000000000003,"5min":4.5579999999999998,"15min":4.5579999999999998},"max":{"1min":16.622,"5min":154.36199999999999,"15min":608.46199999999999},"last":7.3250000000000002}]},{"osd":1,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":45.543999999999997,"5min":20.469000000000001,"15min":43.674999999999997},"min":{"1min":6.3620000000000001,"5min":5.4930000000000003,"15min":5.4930000000000003},"max":{"1min":633.78700000000003,"5min":633.78700000000003,"15min":633.78700000000003},"last":10.199999999999999},{"interface":"front","average":{"1min":46.32,"5min":23.73,"15min":46.381},"min":{"1min":10.032,"5min":5.9000000000000004,"15min":5.8259999999999996},"max":{"1min":598.60000000000002,"5min":598.60000000000002,"15min":701.19399999999996},"last":9.6280000000000001}]},{"osd":2,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":14.177,"5min":14.871,"15min":44.731999999999999},"min":{"1min":8.218,"5min":6.8369999999999997,"15min":6.6029999999999998},"max":{"1min":27.632000000000001,"5min":87.468000000000004,"15min":603.024},"last":12.025},{"interface":"front","average":{"1min":15.343,"5min":17.352,"15min":49.799999999999997},"min":{"1min":10.210000000000001,"5min":8.1289999999999996,"15min":8.1289999999999996},"max":{"1min":29.491,"5min":84.820999999999998,"15min":690.71799999999996},"last":17.169}]},{"osd":3,"last update":"Fri Oct 6 14:26:46 2023","interfaces":[{"interface":"back","average":{"1min":18.097000000000001,"5min":24.934999999999999,"15min":31.651},"min":{"1min":8.0079999999999991,"5min":8.0079999999999991,"15min":8.0079999999999991},"max":{"1min":76.787000000000006,"5min":264.37900000000002,"15min":264.37900000000002},"last":12.803000000000001},{"interface":"front","average":{"1min":18.696000000000002,"5min":24.536000000000001,"15min":29.300000000000001},"min":{"1min":9.5779999999999994,"5min":9.0990000000000002,"15min":9.0990000000000002},"max":{"1min":83.009,"5min":173.858,"15min":173.858},"last":14.630000000000001}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":3491.913,"5min":712.81600000000003,"15min":274.36799999999999},"min":{"1min":10.141,"5min":7.9340000000000002,"15min":7.9340000000000002},"max":{"1min":38256.345000000001,"5min":38256.345000000001,"15min":38256.345000000001},"last":13.880000000000001},{"interface":"front","average":{"1min":3489.7570000000001,"5min":714.00800000000004,"15min":281.79899999999998},"min":{"1min":13.077999999999999,"5min":9.1750000000000007,"15min":8.1839999999999993},"max":{"1min":38222.745999999999,"5min":38222.745999999999,"15min":38222.745999999999},"last":16.440000000000001}]},{"osd":6,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":15.797000000000001,"5min":18.550999999999998,"15min":47.345999999999997},"min":{"1min":10.766,"5min":7.7300000000000004,"15min":7.7300000000000004},"max":{"1min":27.385000000000002,"5min":108.053,"15min":607.87400000000002},"last":15.589},{"interface":"front","average":{"1min":17.838000000000001,"5min":21.890000000000001,"15min":51.551000000000002},"min":{"1min":14.509,"5min":10.855,"15min":7.1040000000000001},"max":{"1min":26.152000000000001,"5min":91.733000000000004,"15min":739.33100000000002},"last":18.594000000000001}]},{"osd":7,"last update":"Fri Oct 6 14:25:53 2023","interfaces":[{"interface":"back","average":{"1min":17.960999999999999,"5min":21.77,"15min":47.536000000000001},"min":{"1min":10.858000000000001,"5min":9.3569999999999993,"15min":7.0350000000000001},"max":{"1min":32.581000000000003,"5min":90.037000000000006,"15min":609.59799999999996},"last":17.895},{"interface":"front","average":{"1min":9.0069999999999997,"5min":13.292,"15min":44.540999999999997},"min":{"1min":6.4420000000000002,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":13.343999999999999,"5min":77.034999999999997,"15min":610.43799999999999},"last":8.6389999999999993}]}]},{"osd":7,"up_from":13,"seq":55834575239,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80884,"kb_used_data":5548,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94290956,"statfs":{"total":96636764160,"available":96553938944,"internally_reserved":0,"allocated":5681152,"data_stored":3019897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:31 2023","interfaces":[{"interface":"back","average":{"1min":8.9030000000000005,"5min":10.318,"15min":32.875999999999998},"min":{"1min":4.8620000000000001,"5min":4.742,"15min":4.2539999999999996},"max":{"1min":23.603000000000002,"5min":46.529000000000003,"15min":930.62099999999998},"last":5.2919999999999998},{"interface":"front","average":{"1min":11.708,"5min":11.972,"15min":31.646999999999998},"min":{"1min":5.1619999999999999,"5min":4.8079999999999998,"15min":4.8079999999999998},"max":{"1min":39.262,"5min":69.477999999999994,"15min":879.35199999999998},"last":6.8929999999999998}]},{"osd":1,"last update":"Fri Oct 6 14:26:31 2023","interfaces":[{"interface":"back","average":{"1min":11.952999999999999,"5min":26.451000000000001,"15min":39.228000000000002},"min":{"1min":6.9059999999999997,"5min":5.4790000000000001,"15min":5.4790000000000001},"max":{"1min":23.271000000000001,"5min":1094.5889999999999,"15min":1094.5889999999999},"last":13.363},{"interface":"front","average":{"1min":12.124000000000001,"5min":25.449000000000002,"15min":24.937000000000001},"min":{"1min":6.2110000000000003,"5min":6.2110000000000003,"15min":5.4539999999999997},"max":{"1min":28.507999999999999,"5min":1057.856,"15min":1057.856},"last":12.528}]},{"osd":2,"last update":"Fri Oct 6 14:26:31 2023","interfaces":[{"interface":"back","average":{"1min":11.727,"5min":15.356,"15min":23.838000000000001},"min":{"1min":6.9969999999999999,"5min":6.5149999999999997,"15min":6.202},"max":{"1min":30.532,"5min":75.823999999999998,"15min":316.84399999999999},"last":14.426},{"interface":"front","average":{"1min":14.474,"5min":16.344000000000001,"15min":39.460000000000001},"min":{"1min":8.4879999999999995,"5min":5.6500000000000004,"15min":5.6500000000000004},"max":{"1min":31.562999999999999,"5min":98.183000000000007,"15min":931.94500000000005},"last":7.7380000000000004}]},{"osd":3,"last update":"Fri Oct 6 14:26:47 2023","interfaces":[{"interface":"back","average":{"1min":11.794,"5min":18.402999999999999,"15min":22.134},"min":{"1min":7.6349999999999998,"5min":6.9969999999999999,"15min":6.9969999999999999},"max":{"1min":16.472999999999999,"5min":184.93700000000001,"15min":184.93700000000001},"last":15.151},{"interface":"front","average":{"1min":14.417999999999999,"5min":17.114000000000001,"15min":18.498999999999999},"min":{"1min":7.54,"5min":7.54,"15min":7.54},"max":{"1min":19.154,"5min":77.144000000000005,"15min":77.144000000000005},"last":19.562000000000001}]},{"osd":4,"last update":"Fri Oct 6 14:26:06 2023","interfaces":[{"interface":"back","average":{"1min":2132.1480000000001,"5min":439.87099999999998,"15min":167.66999999999999},"min":{"1min":7.8869999999999996,"5min":5.9429999999999996,"15min":5.9429999999999996},"max":{"1min":40210.642,"5min":40210.642,"15min":40210.642},"last":16.312000000000001},{"interface":"front","average":{"1min":2144.9340000000002,"5min":442.64400000000001,"15min":163.51599999999999},"min":{"1min":11.497999999999999,"5min":7.4180000000000001,"15min":6.9279999999999999},"max":{"1min":40377.220000000001,"5min":40377.220000000001,"15min":40377.220000000001},"last":17.548999999999999}]},{"osd":5,"last update":"Fri Oct 6 14:26:50 2023","interfaces":[{"interface":"back","average":{"1min":12.994999999999999,"5min":18.428000000000001,"15min":50.965000000000003},"min":{"1min":6.9790000000000001,"5min":6.9790000000000001,"15min":6.9790000000000001},"max":{"1min":17.867000000000001,"5min":149.61799999999999,"15min":6750.3770000000004},"last":18.768999999999998},{"interface":"front","average":{"1min":13.414,"5min":19.297999999999998,"15min":51.856000000000002},"min":{"1min":8.0630000000000006,"5min":7.4009999999999998,"15min":6.9290000000000003},"max":{"1min":19.074000000000002,"5min":174.839,"15min":6875.5780000000004},"last":9.0500000000000007}]},{"osd":6,"last update":"Fri Oct 6 14:26:31 2023","interfaces":[{"interface":"back","average":{"1min":15.701000000000001,"5min":18.128,"15min":26.704000000000001},"min":{"1min":10.023,"5min":7.9509999999999996,"15min":7.7290000000000001},"max":{"1min":36.390000000000001,"5min":75.013999999999996,"15min":380.74000000000001},"last":16.920999999999999},{"interface":"front","average":{"1min":18.978000000000002,"5min":20.349,"15min":44.131},"min":{"1min":9.2360000000000007,"5min":8.9489999999999998,"15min":8.9489999999999998},"max":{"1min":82.266999999999996,"5min":120.83799999999999,"15min":935.49300000000005},"last":8.4789999999999992}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T14:26:58.150 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:58.150 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2023-10-06T13:54:15.007539+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:58.150 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-06T13:53:14.910188+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=53, tm_sec=14, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:58.151 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-06T13:53:14.910188+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=53, tm_sec=14, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:58.151 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-06T13:53:14.910188+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=53, tm_sec=14, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:58.151 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T13:53:14.910188+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=53, tm_sec=14, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:58.151 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-06T13:53:14.910188+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=13, tm_min=53, tm_sec=14, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=14, tm_min=26, tm_sec=27, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T14:26:58.152 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T14:27:08.601 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:11.723 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:08.605 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:11.728 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:08.620 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:11.743 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:08.646 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:11.743 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:08.660 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:11.745 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:08.660 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:11.772 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:08.660 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:11.775 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:08.666 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:11.763 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:08.671 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:11.794 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:18.152 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T14:27:18.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.292+0000 7f9edc50a640 10 monclient: get_monmap_and_config 2023-10-06T14:27:18.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.292+0000 7f9edc50a640 10 monclient: build_initial_monmap 2023-10-06T14:27:18.294 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.292+0000 7f9edc50a640 10 monclient: monmap: 2023-10-06T14:27:18.294 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:27:18.294 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:27:18.294 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:27:18.293220+0000 2023-10-06T14:27:18.294 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:27:18.293220+0000 2023-10-06T14:27:18.295 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:27:18.295 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:27:18.295 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:27:18.295 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:27:18.295 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:27:18.295 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:27:18.295 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:27:18.295 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:27:18.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.292+0000 7f9edc50a640 1 Processor -- start 2023-10-06T14:27:18.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.292+0000 7f9edc50a640 1 -- start start 2023-10-06T14:27:18.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.292+0000 7f9edc50a640 10 monclient: init 2023-10-06T14:27:18.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.293+0000 7f9edc50a640 10 monclient: _reopen_session rank -1 2023-10-06T14:27:18.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.293+0000 7f9edc50a640 10 monclient: _add_conns ranks=[2,1,3,4,0] 2023-10-06T14:27:18.297 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.293+0000 7f9edc50a640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9ed406b6d0 0x7f9ed4144200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:27:18.298 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.293+0000 7f9edc50a640 10 monclient(hunting): picked mon.noname-b con 0x7f9ed406b6d0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:27:18.298 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.293+0000 7f9edc50a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9ed4144740 addr v1:172.21.15.83:6789/0 2023-10-06T14:27:18.298 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.293+0000 7f9edc50a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9ed413e630 0x7f9ed414b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:27:18.298 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.293+0000 7f9edc50a640 10 monclient(hunting): picked mon.noname-a con 0x7f9ed413e630 addr v2:172.21.15.19:3300/0 2023-10-06T14:27:18.298 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.293+0000 7f9edc50a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed406baa0 con 0x7f9ed4144740 2023-10-06T14:27:18.298 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.293+0000 7f9edc50a640 10 monclient(hunting): start opening mon connection 2023-10-06T14:27:18.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.293+0000 7f9edc50a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed413ea00 con 0x7f9ed413e630 2023-10-06T14:27:18.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.293+0000 7f9edc50a640 10 monclient(hunting): start opening mon connection 2023-10-06T14:27:18.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.293+0000 7f9edc50a640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40b2530 con 0x7f9ed406b6d0 2023-10-06T14:27:18.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.293+0000 7f9edc50a640 10 monclient(hunting): _renew_subs 2023-10-06T14:27:18.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.293+0000 7f9edc50a640 10 monclient(hunting): authenticate will time out at 3366.479969s 2023-10-06T14:27:18.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.294+0000 7f9eda2a6640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9ed406b6d0 0x7f9ed4144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:27:18.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.294+0000 7f9ed9aa5640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9ed4144740 0x7f9ed413df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.19:37938/0 (socket says 172.21.15.19:37938) 2023-10-06T14:27:18.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.294+0000 7f9ed9aa5640 1 -- 172.21.15.19:0/3064894088 learned_addr learned my addr 172.21.15.19:0/3064894088 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-10-06T14:27:18.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.294+0000 7f9edaaa7640 1 --2- 172.21.15.19:0/3064894088 >> v2:172.21.15.19:3300/0 conn(0x7f9ed413e630 0x7f9ed414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:27:18.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.294+0000 7f9eda2a6640 10 monclient(hunting): get_auth_request con 0x7f9ed406b6d0 auth_method 0 2023-10-06T14:27:18.299 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.294+0000 7f9eda2a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:27:18.300 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.294+0000 7f9eda2a6640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:27:18.300 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.294+0000 7f9eda2a6640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:27:18.300 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.294+0000 7f9edaaa7640 10 monclient(hunting): get_auth_request con 0x7f9ed413e630 auth_method 0 2023-10-06T14:27:18.300 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.294+0000 7f9edaaa7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:27:18.300 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.294+0000 7f9edaaa7640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:27:18.300 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.294+0000 7f9edaaa7640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:27:18.300 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.295+0000 7f9eda2a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:27:18.300 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.295+0000 7f9eda2a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:27:18.300 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.295+0000 7f9eda2a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:27:18.300 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.295+0000 7f9edaaa7640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:27:18.300 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.295+0000 7f9edaaa7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:27:18.300 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.295+0000 7f9edaaa7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:27:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9ed92a4640 1 -- 172.21.15.19:0/3064894088 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735816706 0 0) 0x7f9ed406baa0 con 0x7f9ed4144740 2023-10-06T14:27:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9ed92a4640 10 monclient(hunting): _negotiate 2023-10-06T14:27:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9ed92a4640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:27:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9ed92a4640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:27:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9ed92a4640 10 monclient(hunting): my global_id is 20833 2023-10-06T14:27:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9ed92a4640 1 -- 172.21.15.19:0/3064894088 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ec0003520 con 0x7f9ed4144740 2023-10-06T14:27:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9eda2a6640 10 monclient(hunting): handle_auth_done global_id 17222 payload 274 2023-10-06T14:27:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9eda2a6640 1 -- 172.21.15.19:0/3064894088 >> v2:172.21.15.19:3300/0 conn(0x7f9ed413e630 msgr2=0x7f9ed414b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:27:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9eda2a6640 1 --2- 172.21.15.19:0/3064894088 >> v2:172.21.15.19:3300/0 conn(0x7f9ed413e630 0x7f9ed414b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9eda2a6640 1 -- 172.21.15.19:0/3064894088 >> v1:172.21.15.83:6789/0 conn(0x7f9ed4144740 legacy=0x7f9ed413df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:27:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9eda2a6640 10 monclient: _finish_hunting 0 2023-10-06T14:27:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9eda2a6640 1 monclient: found mon.noname-b 2023-10-06T14:27:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9eda2a6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:27:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9eda2a6640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T14:27:18.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.296+0000 7f9eda2a6640 1 -- 172.21.15.19:0/3064894088 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40b2810 con 0x7f9ed406b6d0 2023-10-06T14:27:18.303 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.298+0000 7f9eda2a6640 1 --2- 172.21.15.19:0/3064894088 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9ed406b6d0 0x7f9ed4144200 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00097a0 tx=0x7f9ed002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86b52736a3708ef9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:27:18.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9ed92a4640 1 -- 172.21.15.19:0/3064894088 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0031830 con 0x7f9ed406b6d0 2023-10-06T14:27:18.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9ed92a4640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:27:18.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9ed92a4640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:27:18.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9ed92a4640 10 monclient: dump: 2023-10-06T14:27:18.304 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:27:18.304 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:27:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:27:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:27:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:27:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:27:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:27:18.305 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:27:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9ed92a4640 10 monclient: _finish_auth 0 2023-10-06T14:27:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9ed92a4640 10 monclient: _check_auth_tickets 2023-10-06T14:27:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9ed92a4640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:27:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9ed92a4640 1 -- 172.21.15.19:0/3064894088 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed0031990 con 0x7f9ed406b6d0 2023-10-06T14:27:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9edc50a640 5 monclient: authenticate success, global_id 17222 2023-10-06T14:27:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9ed92a4640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:27:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9ed92a4640 1 -- 172.21.15.19:0/3064894088 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0031d20 con 0x7f9ed406b6d0 2023-10-06T14:27:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9edc50a640 10 monclient: get_monmap_and_config success 2023-10-06T14:27:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9edc50a640 4 set_mon_vals no callback set 2023-10-06T14:27:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9ed92a4640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:27:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9ed92a4640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:27:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9ed92a4640 10 monclient: dump: 2023-10-06T14:27:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:27:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:27:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:27:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:27:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:27:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:27:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:27:18.307 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:27:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9edc50a640 10 monclient: shutdown 2023-10-06T14:27:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9edc50a640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:27:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9edc50a640 1 -- 172.21.15.19:0/3064894088 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9ed406b6d0 msgr2=0x7f9ed4144200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:27:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.302+0000 7f9edc50a640 1 --2- 172.21.15.19:0/3064894088 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9ed406b6d0 0x7f9ed4144200 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00097a0 tx=0x7f9ed002b8d0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.303+0000 7f9edc50a640 1 -- 172.21.15.19:0/3064894088 shutdown_connections 2023-10-06T14:27:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.303+0000 7f9edc50a640 1 --2- 172.21.15.19:0/3064894088 >> v2:172.21.15.19:3300/0 conn(0x7f9ed413e630 0x7f9ed414b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.303+0000 7f9edc50a640 1 --2- 172.21.15.19:0/3064894088 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9ed406b6d0 0x7f9ed4144200 unknown :-1 s=CLOSED pgs=3133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.303+0000 7f9edc50a640 1 -- 172.21.15.19:0/3064894088 >> 172.21.15.19:0/3064894088 conn(0x7f9ed40665b0 msgr2=0x7f9ed406ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:27:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.303+0000 7f9edc50a640 1 -- 172.21.15.19:0/3064894088 shutdown_connections 2023-10-06T14:27:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.303+0000 7f9edc50a640 1 -- 172.21.15.19:0/3064894088 wait complete. 2023-10-06T14:27:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.303+0000 7f9edc50a640 10 monclient: build_initial_monmap 2023-10-06T14:27:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.303+0000 7f9edc50a640 10 monclient: monmap: 2023-10-06T14:27:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:27:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:27:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:27:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:27:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:27:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:27:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:27:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:27:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:27:18.312 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:27:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.303+0000 7f9edc50a640 1 librados: starting msgr at 2023-10-06T14:27:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 1 librados: starting objecter 2023-10-06T14:27:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 1 Processor -- start 2023-10-06T14:27:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 1 -- start start 2023-10-06T14:27:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 1 librados: setting wanted keys 2023-10-06T14:27:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 1 librados: calling monclient init 2023-10-06T14:27:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 10 monclient: init 2023-10-06T14:27:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 10 monclient: _reopen_session rank -1 2023-10-06T14:27:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 10 monclient: _add_conns ranks=[2,1,0] 2023-10-06T14:27:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9ed406b6d0 0x7f9ed41d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:27:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 10 monclient(hunting): picked mon.noname-c con 0x7f9ed406b6d0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:27:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ed413e630 0x7f9ed4137f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:27:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 10 monclient(hunting): picked mon.noname-b con 0x7f9ed413e630 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:27:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4144740 0x7f9ed4138490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:27:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 10 monclient(hunting): picked mon.noname-a con 0x7f9ed4144740 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:27:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 10 monclient(hunting): start opening mon connection 2023-10-06T14:27:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed41d5740 con 0x7f9ed4144740 2023-10-06T14:27:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 10 monclient(hunting): start opening mon connection 2023-10-06T14:27:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed41389d0 con 0x7f9ed413e630 2023-10-06T14:27:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 10 monclient(hunting): start opening mon connection 2023-10-06T14:27:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed413fa40 con 0x7f9ed406b6d0 2023-10-06T14:27:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 10 monclient(hunting): _renew_subs 2023-10-06T14:27:18.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.304+0000 7f9edc50a640 10 monclient(hunting): authenticate will time out at 3366.491324s 2023-10-06T14:27:18.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.305+0000 7f9edaaa7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4144740 0x7f9ed4138490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:27:18.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.305+0000 7f9edaaa7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4144740 0x7f9ed4138490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33360/0 (socket says 172.21.15.19:33360) 2023-10-06T14:27:18.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.305+0000 7f9edaaa7640 1 -- 172.21.15.19:0/1695714626 learned_addr learned my addr 172.21.15.19:0/1695714626 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:27:18.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.305+0000 7f9edaaa7640 10 monclient(hunting): get_auth_request con 0x7f9ed4144740 auth_method 0 2023-10-06T14:27:18.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.305+0000 7f9edaaa7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:27:18.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.305+0000 7f9edaaa7640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:27:18.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.305+0000 7f9edaaa7640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:27:18.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.306+0000 7f9eda2a6640 1 --2- 172.21.15.19:0/1695714626 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9ed406b6d0 0x7f9ed41d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:27:18.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.306+0000 7f9edaaa7640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:27:18.314 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.306+0000 7f9edaaa7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:27:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.306+0000 7f9edaaa7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:27:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.306+0000 7f9eda2a6640 10 monclient(hunting): get_auth_request con 0x7f9ed406b6d0 auth_method 0 2023-10-06T14:27:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.306+0000 7f9eda2a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:27:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.306+0000 7f9eda2a6640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:27:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.306+0000 7f9eda2a6640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:27:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.307+0000 7f9ed9aa5640 1 --2- 172.21.15.19:0/1695714626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ed413e630 0x7f9ed4137f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:27:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.307+0000 7f9eda2a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:27:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.307+0000 7f9eda2a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:27:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.307+0000 7f9eda2a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:27:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.307+0000 7f9ed9aa5640 10 monclient(hunting): get_auth_request con 0x7f9ed413e630 auth_method 0 2023-10-06T14:27:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.307+0000 7f9ed9aa5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:27:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.307+0000 7f9ed9aa5640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:27:18.316 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.307+0000 7f9ed9aa5640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:27:18.317 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.309+0000 7f9ed9aa5640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:27:18.317 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.309+0000 7f9ed9aa5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:27:18.317 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.309+0000 7f9ed9aa5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:27:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.309+0000 7f9edaaa7640 10 monclient(hunting): handle_auth_done global_id 15720 payload 824 2023-10-06T14:27:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.309+0000 7f9edaaa7640 1 -- 172.21.15.19:0/1695714626 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9ed406b6d0 msgr2=0x7f9ed41d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:27:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.309+0000 7f9edaaa7640 1 --2- 172.21.15.19:0/1695714626 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9ed406b6d0 0x7f9ed41d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.309+0000 7f9edaaa7640 1 -- 172.21.15.19:0/1695714626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ed413e630 msgr2=0x7f9ed4137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:27:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.309+0000 7f9edaaa7640 1 --2- 172.21.15.19:0/1695714626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ed413e630 0x7f9ed4137f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.309+0000 7f9edaaa7640 10 monclient: _finish_hunting 0 2023-10-06T14:27:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.309+0000 7f9edaaa7640 1 monclient: found mon.noname-a 2023-10-06T14:27:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.309+0000 7f9edaaa7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:27:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.309+0000 7f9edaaa7640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-10-06T14:27:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.309+0000 7f9edaaa7640 1 -- 172.21.15.19:0/1695714626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed0009400 con 0x7f9ed4144740 2023-10-06T14:27:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.312+0000 7f9edaaa7640 1 --2- 172.21.15.19:0/1695714626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4144740 0x7f9ed4138490 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009dd0 tx=0x7f9ecc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:27:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.312+0000 7f9ecaffd640 10 client.?.objecter ms_handle_connect 0x7f9ed4144740 2023-10-06T14:27:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.312+0000 7f9ecaffd640 10 client.?.objecter resend_mon_ops 2023-10-06T14:27:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9ecaffd640 1 -- 172.21.15.19:0/1695714626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc014480 con 0x7f9ed4144740 2023-10-06T14:27:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9ecaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:27:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9ecaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-10-06T14:27:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9ecaffd640 10 monclient: dump: 2023-10-06T14:27:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:27:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:27:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:27:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:27:18.320 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:27:18.320 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:27:18.320 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:27:18.320 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:27:18.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9ecaffd640 10 monclient: _finish_auth 0 2023-10-06T14:27:18.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9ecaffd640 10 monclient: _check_auth_tickets 2023-10-06T14:27:18.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9ecaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:27:18.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9edc50a640 5 monclient: authenticate success, global_id 15720 2023-10-06T14:27:18.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9edc50a640 10 monclient: _renew_subs 2023-10-06T14:27:18.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9edc50a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T14:27:18.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9edc50a640 1 -- 172.21.15.19:0/1695714626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed40aa560 con 0x7f9ed4144740 2023-10-06T14:27:18.320 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9edc50a640 10 client.15720.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:27:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9edc50a640 10 monclient: _renew_subs 2023-10-06T14:27:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9edc50a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T14:27:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9edc50a640 1 -- 172.21.15.19:0/1695714626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed41d62f0 con 0x7f9ed4144740 2023-10-06T14:27:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9edc50a640 1 librados: init done 2023-10-06T14:27:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9ecaffd640 1 -- 172.21.15.19:0/1695714626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ecc00f040 con 0x7f9ed4144740 2023-10-06T14:27:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9ecaffd640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:27:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9ed8aa3640 4 set_mon_vals no callback set 2023-10-06T14:27:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9ecaffd640 1 -- 172.21.15.19:0/1695714626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc0147e0 con 0x7f9ed4144740 2023-10-06T14:27:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9ecaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:27:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9ecaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-10-06T14:27:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.315+0000 7f9ecaffd640 10 monclient: dump: 2023-10-06T14:27:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:27:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:27:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:27:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:27:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:27:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:27:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:27:18.322 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:27:18.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.316+0000 7f9edc50a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:27:18.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.316+0000 7f9edc50a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:27:18.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.316+0000 7f9edc50a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-10-06T14:27:18.323 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.316+0000 7f9edc50a640 1 -- 172.21.15.19:0/1695714626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed404e000 con 0x7f9ed4144740 2023-10-06T14:27:18.584 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.581+0000 7f9ecaffd640 1 -- 172.21.15.19:0/1695714626 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f9ecc014940 con 0x7f9ed4144740 2023-10-06T14:27:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.582+0000 7f9ecaffd640 1 --2- 172.21.15.19:0/1695714626 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9eb806c1c0 0x7f9eb806e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:27:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.582+0000 7f9eda2a6640 1 --2- 172.21.15.19:0/1695714626 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9eb806c1c0 0x7f9eb806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:27:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.582+0000 7f9eda2a6640 10 monclient: get_auth_request con 0x7f9eb806c1c0 auth_method 0 2023-10-06T14:27:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.582+0000 7f9eda2a6640 1 --2- 172.21.15.19:0/1695714626 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9eb806c1c0 0x7f9eb806e670 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0036000 tx=0x7f9ed003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:27:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.582+0000 7f9ecaffd640 10 client.15720.objecter ms_handle_connect 0x7f9eb806c1c0 2023-10-06T14:27:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.606+0000 7f9ecaffd640 1 -- 172.21.15.19:0/1695714626 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(970..970 src has 714..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f9ecc059220 con 0x7f9ed4144740 2023-10-06T14:27:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.606+0000 7f9ecaffd640 10 client.15720.objecter ms_dispatch 0x7f9ed4000b90 osd_map(970..970 src has 714..970) v4 2023-10-06T14:27:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.606+0000 7f9ecaffd640 3 client.15720.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:27:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.606+0000 7f9ecaffd640 3 client.15720.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:27:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.606+0000 7f9ecaffd640 20 client.15720.objecter dump_active .. 0 homeless 2023-10-06T14:27:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.606+0000 7f9ecaffd640 1 -- 172.21.15.19:0/1695714626 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f9ecc055560 con 0x7f9ed4144740 2023-10-06T14:27:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.606+0000 7f9ecaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:27:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.606+0000 7f9ecaffd640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:27:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.785+0000 7f9edc50a640 1 -- 172.21.15.19:0/1695714626 --> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ed41d55a0 con 0x7f9eb806c1c0 2023-10-06T14:27:18.792 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.790+0000 7f9ecaffd640 1 -- 172.21.15.19:0/1695714626 <== mgr.4099 v2:172.21.15.83:6800/85581 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152389 (secure 0 0 0) 0x7f9ed0031630 con 0x7f9eb806c1c0 2023-10-06T14:27:18.794 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-10-06T14:27:18.795 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2023-10-06T14:27:18.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 10 librados: watch_flush enter 2023-10-06T14:27:18.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 10 librados: watch_flush exit 2023-10-06T14:27:18.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 20 client.15720.objecter shutdown clearing up homeless session... 2023-10-06T14:27:18.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 10 client.15720.objecter successfully canceled tick 2023-10-06T14:27:18.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 1 -- 172.21.15.19:0/1695714626 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9eb806c1c0 msgr2=0x7f9eb806e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:27:18.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 1 --2- 172.21.15.19:0/1695714626 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9eb806c1c0 0x7f9eb806e670 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0036000 tx=0x7f9ed003c040 comp rx=0 tx=0).stop 2023-10-06T14:27:18.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 10 monclient: shutdown 2023-10-06T14:27:18.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:27:18.797 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 1 -- 172.21.15.19:0/1695714626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4144740 msgr2=0x7f9ed4138490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:27:18.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 1 --2- 172.21.15.19:0/1695714626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4144740 0x7f9ed4138490 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009dd0 tx=0x7f9ecc00c490 comp rx=0 tx=0).stop 2023-10-06T14:27:18.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 1 -- 172.21.15.19:0/1695714626 shutdown_connections 2023-10-06T14:27:18.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 1 --2- 172.21.15.19:0/1695714626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4144740 0x7f9ed4138490 unknown :-1 s=CLOSED pgs=3204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 1 --2- 172.21.15.19:0/1695714626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ed413e630 0x7f9ed4137f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 1 --2- 172.21.15.19:0/1695714626 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f9eb806c1c0 0x7f9eb806e670 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 1 --2- 172.21.15.19:0/1695714626 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f9ed406b6d0 0x7f9ed41d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.798 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 1 -- 172.21.15.19:0/1695714626 >> 172.21.15.19:0/1695714626 conn(0x7f9ed40665b0 msgr2=0x7f9ed40b75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:27:18.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 1 -- 172.21.15.19:0/1695714626 shutdown_connections 2023-10-06T14:27:18.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 1 -- 172.21.15.19:0/1695714626 wait complete. 2023-10-06T14:27:18.799 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.792+0000 7f9edc50a640 1 librados: shutdown 2023-10-06T14:27:18.805 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":1999,"stamp":"2023-10-06T14:27:17.816508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":515000,"kb_used_data":45176,"kb_used_omap":0,"kb_used_meta":469760,"kb_avail":754459720,"statfs":{"total":773094113280,"available":772566753280,"internally_reserved":0,"allocated":46260224,"data_stored":24954866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":481034240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003045"},"pg_stats":[{"pgid":"8.14","version":"0'0","reported_seq":936,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:36.505925+0000","last_change":"2023-10-06T14:26:36.505925+0000","last_active":"2023-10-06T14:26:36.505925+0000","last_peered":"2023-10-06T14:26:36.505925+0000","last_clean":"2023-10-06T14:26:36.505925+0000","last_became_active":"2023-10-06T13:54:27.664679+0000","last_became_peered":"2023-10-06T13:54:27.664679+0000","last_unstale":"2023-10-06T14:26:36.505925+0000","last_undegraded":"2023-10-06T14:26:36.505925+0000","last_fullsized":"2023-10-06T14:26:36.505925+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:36.502985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:36.502985+0000","last_clean_scrub_stamp":"2023-10-06T14:26:36.502985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:28.334974+0000","scrub_duration":0.037170340000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":934,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:27:00.529546+0000","last_change":"2023-10-06T14:27:00.531411+0000","last_active":"2023-10-06T14:27:00.529546+0000","last_peered":"2023-10-06T14:27:00.529546+0000","last_clean":"2023-10-06T14:27:00.529546+0000","last_became_active":"2023-10-06T14:22:39.724017+0000","last_became_peered":"2023-10-06T14:22:39.724017+0000","last_unstale":"2023-10-06T14:27:00.529546+0000","last_undegraded":"2023-10-06T14:27:00.529546+0000","last_fullsized":"2023-10-06T14:27:00.529546+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:27:00.528700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:27:00.528700+0000","last_clean_scrub_stamp":"2023-10-06T14:27:00.528700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:06:28.911160+0000","scrub_duration":0.068722571999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":939,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:47.942073+0000","last_change":"2023-10-06T14:26:47.947183+0000","last_active":"2023-10-06T14:26:47.942073+0000","last_peered":"2023-10-06T14:26:47.942073+0000","last_clean":"2023-10-06T14:26:47.942073+0000","last_became_active":"2023-10-06T14:16:33.542923+0000","last_became_peered":"2023-10-06T14:16:33.542923+0000","last_unstale":"2023-10-06T14:26:47.942073+0000","last_undegraded":"2023-10-06T14:26:47.942073+0000","last_fullsized":"2023-10-06T14:26:47.942073+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:47.941299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:47.941299+0000","last_clean_scrub_stamp":"2023-10-06T14:26:47.941299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:42:17.558417+0000","scrub_duration":0.046222051,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":941,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:47.724203+0000","last_change":"2023-10-06T14:26:47.729390+0000","last_active":"2023-10-06T14:26:47.724203+0000","last_peered":"2023-10-06T14:26:47.724203+0000","last_clean":"2023-10-06T14:26:47.724203+0000","last_became_active":"2023-10-06T13:54:29.163161+0000","last_became_peered":"2023-10-06T13:54:29.163161+0000","last_unstale":"2023-10-06T14:26:47.724203+0000","last_undegraded":"2023-10-06T14:26:47.724203+0000","last_fullsized":"2023-10-06T14:26:47.724203+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:47.723383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:47.723383+0000","last_clean_scrub_stamp":"2023-10-06T14:26:47.723383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:06:07.573686+0000","scrub_duration":0.070208072999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":933,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:54.600485+0000","last_change":"2023-10-06T14:26:54.605997+0000","last_active":"2023-10-06T14:26:54.600485+0000","last_peered":"2023-10-06T14:26:54.600485+0000","last_clean":"2023-10-06T14:26:54.600485+0000","last_became_active":"2023-10-06T14:22:39.128927+0000","last_became_peered":"2023-10-06T14:22:39.128927+0000","last_unstale":"2023-10-06T14:26:54.600485+0000","last_undegraded":"2023-10-06T14:26:54.600485+0000","last_fullsized":"2023-10-06T14:26:54.600485+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:54.599663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:54.599663+0000","last_clean_scrub_stamp":"2023-10-06T14:26:54.599663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:41.366981+0000","scrub_duration":0.032689358000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":934,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:39.527703+0000","last_change":"2023-10-06T14:26:39.533028+0000","last_active":"2023-10-06T14:26:39.527703+0000","last_peered":"2023-10-06T14:26:39.527703+0000","last_clean":"2023-10-06T14:26:39.527703+0000","last_became_active":"2023-10-06T14:22:39.418221+0000","last_became_peered":"2023-10-06T14:22:39.418221+0000","last_unstale":"2023-10-06T14:26:39.527703+0000","last_undegraded":"2023-10-06T14:26:39.527703+0000","last_fullsized":"2023-10-06T14:26:39.527703+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:39.526884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:39.526884+0000","last_clean_scrub_stamp":"2023-10-06T14:26:39.526884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:54:10.734371+0000","scrub_duration":0.069040852,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":934,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:49.532283+0000","last_change":"2023-10-06T14:26:49.534206+0000","last_active":"2023-10-06T14:26:49.532283+0000","last_peered":"2023-10-06T14:26:49.532283+0000","last_clean":"2023-10-06T14:26:49.532283+0000","last_became_active":"2023-10-06T14:22:39.195040+0000","last_became_peered":"2023-10-06T14:22:39.195040+0000","last_unstale":"2023-10-06T14:26:49.532283+0000","last_undegraded":"2023-10-06T14:26:49.532283+0000","last_fullsized":"2023-10-06T14:26:49.532283+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:49.531420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:49.531420+0000","last_clean_scrub_stamp":"2023-10-06T14:26:49.531420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:38.168606+0000","scrub_duration":0.058446350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":940,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:44.019889+0000","last_change":"2023-10-06T14:26:44.021646+0000","last_active":"2023-10-06T14:26:44.019889+0000","last_peered":"2023-10-06T14:26:44.019889+0000","last_clean":"2023-10-06T14:26:44.019889+0000","last_became_active":"2023-10-06T14:16:33.300222+0000","last_became_peered":"2023-10-06T14:16:33.300222+0000","last_unstale":"2023-10-06T14:26:44.019889+0000","last_undegraded":"2023-10-06T14:26:44.019889+0000","last_fullsized":"2023-10-06T14:26:44.019889+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:44.019035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:44.019035+0000","last_clean_scrub_stamp":"2023-10-06T14:26:44.019035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:08.569157+0000","scrub_duration":0.061160045000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":933,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:50.531352+0000","last_change":"2023-10-06T14:26:50.533277+0000","last_active":"2023-10-06T14:26:50.531352+0000","last_peered":"2023-10-06T14:26:50.531352+0000","last_clean":"2023-10-06T14:26:50.531352+0000","last_became_active":"2023-10-06T14:22:39.712413+0000","last_became_peered":"2023-10-06T14:22:39.712413+0000","last_unstale":"2023-10-06T14:26:50.531352+0000","last_undegraded":"2023-10-06T14:26:50.531352+0000","last_fullsized":"2023-10-06T14:26:50.531352+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:50.530490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:50.530490+0000","last_clean_scrub_stamp":"2023-10-06T14:26:50.530490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:59.318217+0000","scrub_duration":0.045610221999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":922,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:43.874971+0000","last_change":"2023-10-06T14:26:43.874971+0000","last_active":"2023-10-06T14:26:43.874971+0000","last_peered":"2023-10-06T14:26:43.874971+0000","last_clean":"2023-10-06T14:26:43.874971+0000","last_became_active":"2023-10-06T13:54:19.427005+0000","last_became_peered":"2023-10-06T13:54:19.427005+0000","last_unstale":"2023-10-06T14:26:43.874971+0000","last_undegraded":"2023-10-06T14:26:43.874971+0000","last_fullsized":"2023-10-06T14:26:43.874971+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:43.866518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:43.866518+0000","last_clean_scrub_stamp":"2023-10-06T14:26:43.866518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:08.741049+0000","scrub_duration":0.053312574000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":924,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:47.113658+0000","last_change":"2023-10-06T14:26:47.115292+0000","last_active":"2023-10-06T14:26:47.113658+0000","last_peered":"2023-10-06T14:26:47.113658+0000","last_clean":"2023-10-06T14:26:47.113658+0000","last_became_active":"2023-10-06T13:54:19.531077+0000","last_became_peered":"2023-10-06T13:54:19.531077+0000","last_unstale":"2023-10-06T14:26:47.113658+0000","last_undegraded":"2023-10-06T14:26:47.113658+0000","last_fullsized":"2023-10-06T14:26:47.113658+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:47.112914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:47.112914+0000","last_clean_scrub_stamp":"2023-10-06T14:26:47.112914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:19.752892+0000","scrub_duration":0.045651023999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":934,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:40.376656+0000","last_change":"2023-10-06T14:26:40.378249+0000","last_active":"2023-10-06T14:26:40.376656+0000","last_peered":"2023-10-06T14:26:40.376656+0000","last_clean":"2023-10-06T14:26:40.376656+0000","last_became_active":"2023-10-06T14:06:28.630618+0000","last_became_peered":"2023-10-06T14:06:28.630618+0000","last_unstale":"2023-10-06T14:26:40.376656+0000","last_undegraded":"2023-10-06T14:26:40.376656+0000","last_fullsized":"2023-10-06T14:26:40.376656+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:40.375936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:40.375936+0000","last_clean_scrub_stamp":"2023-10-06T14:26:40.375936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:38:22.194622+0000","scrub_duration":0.067338567000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":943,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:35.321577+0000","last_change":"2023-10-06T14:26:35.327110+0000","last_active":"2023-10-06T14:26:35.321577+0000","last_peered":"2023-10-06T14:26:35.321577+0000","last_clean":"2023-10-06T14:26:35.321577+0000","last_became_active":"2023-10-06T13:54:21.813915+0000","last_became_peered":"2023-10-06T13:54:21.813915+0000","last_unstale":"2023-10-06T14:26:35.321577+0000","last_undegraded":"2023-10-06T14:26:35.321577+0000","last_fullsized":"2023-10-06T14:26:35.321577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:35.320741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:35.320741+0000","last_clean_scrub_stamp":"2023-10-06T14:26:35.320741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:23.967871+0000","scrub_duration":0.060312161000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":937,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:27:06.472419+0000","last_change":"2023-10-06T14:27:06.474221+0000","last_active":"2023-10-06T14:27:06.472419+0000","last_peered":"2023-10-06T14:27:06.472419+0000","last_clean":"2023-10-06T14:27:06.472419+0000","last_became_active":"2023-10-06T14:22:39.673285+0000","last_became_peered":"2023-10-06T14:22:39.673285+0000","last_unstale":"2023-10-06T14:27:06.472419+0000","last_undegraded":"2023-10-06T14:27:06.472419+0000","last_fullsized":"2023-10-06T14:27:06.472419+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:27:06.471597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:27:06.471597+0000","last_clean_scrub_stamp":"2023-10-06T14:27:06.471597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:34.381619+0000","scrub_duration":0.032839305999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":928,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:36.928323+0000","last_change":"2023-10-06T14:26:36.929962+0000","last_active":"2023-10-06T14:26:36.928323+0000","last_peered":"2023-10-06T14:26:36.928323+0000","last_clean":"2023-10-06T14:26:36.928323+0000","last_became_active":"2023-10-06T14:16:33.392050+0000","last_became_peered":"2023-10-06T14:16:33.392050+0000","last_unstale":"2023-10-06T14:26:36.928323+0000","last_undegraded":"2023-10-06T14:26:36.928323+0000","last_fullsized":"2023-10-06T14:26:36.928323+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:36.927606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:36.927606+0000","last_clean_scrub_stamp":"2023-10-06T14:26:36.927606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:29.494165+0000","scrub_duration":0.034420503999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":938,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:37.255284+0000","last_change":"2023-10-06T14:26:37.255284+0000","last_active":"2023-10-06T14:26:37.255284+0000","last_peered":"2023-10-06T14:26:37.255284+0000","last_clean":"2023-10-06T14:26:37.255284+0000","last_became_active":"2023-10-06T14:05:13.248166+0000","last_became_peered":"2023-10-06T14:05:13.248166+0000","last_unstale":"2023-10-06T14:26:37.255284+0000","last_undegraded":"2023-10-06T14:26:37.255284+0000","last_fullsized":"2023-10-06T14:26:37.255284+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:37.252750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:37.252750+0000","last_clean_scrub_stamp":"2023-10-06T14:26:37.252750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:56:31.690711+0000","scrub_duration":0.032023376999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":921,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:46.151441+0000","last_change":"2023-10-06T14:26:46.155677+0000","last_active":"2023-10-06T14:26:46.151441+0000","last_peered":"2023-10-06T14:26:46.151441+0000","last_clean":"2023-10-06T14:26:46.151441+0000","last_became_active":"2023-10-06T13:54:26.218851+0000","last_became_peered":"2023-10-06T13:54:26.218851+0000","last_unstale":"2023-10-06T14:26:46.151441+0000","last_undegraded":"2023-10-06T14:26:46.151441+0000","last_fullsized":"2023-10-06T14:26:46.151441+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:46.150724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:46.150724+0000","last_clean_scrub_stamp":"2023-10-06T14:26:46.150724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:05:58.492416+0000","scrub_duration":0.072809143000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":953,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:27:05.436093+0000","last_change":"2023-10-06T14:27:05.441602+0000","last_active":"2023-10-06T14:27:05.436093+0000","last_peered":"2023-10-06T14:27:05.436093+0000","last_clean":"2023-10-06T14:27:05.436093+0000","last_became_active":"2023-10-06T14:22:39.109893+0000","last_became_peered":"2023-10-06T14:22:39.109893+0000","last_unstale":"2023-10-06T14:27:05.436093+0000","last_undegraded":"2023-10-06T14:27:05.436093+0000","last_fullsized":"2023-10-06T14:27:05.436093+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:27:05.435274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:27:05.435274+0000","last_clean_scrub_stamp":"2023-10-06T14:27:05.435274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:09:55.190202+0000","scrub_duration":0.030999663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":940,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:46.934884+0000","last_change":"2023-10-06T14:26:46.939646+0000","last_active":"2023-10-06T14:26:46.934884+0000","last_peered":"2023-10-06T14:26:46.934884+0000","last_clean":"2023-10-06T14:26:46.934884+0000","last_became_active":"2023-10-06T14:16:33.551586+0000","last_became_peered":"2023-10-06T14:16:33.551586+0000","last_unstale":"2023-10-06T14:26:46.934884+0000","last_undegraded":"2023-10-06T14:26:46.934884+0000","last_fullsized":"2023-10-06T14:26:46.934884+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:46.934161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:46.934161+0000","last_clean_scrub_stamp":"2023-10-06T14:26:46.934161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:55:04.465763+0000","scrub_duration":0.048811647999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":934,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:34.944308+0000","last_change":"2023-10-06T14:26:34.944308+0000","last_active":"2023-10-06T14:26:34.944308+0000","last_peered":"2023-10-06T14:26:34.944308+0000","last_clean":"2023-10-06T14:26:34.944308+0000","last_became_active":"2023-10-06T14:16:33.398349+0000","last_became_peered":"2023-10-06T14:16:33.398349+0000","last_unstale":"2023-10-06T14:26:34.944308+0000","last_undegraded":"2023-10-06T14:26:34.944308+0000","last_fullsized":"2023-10-06T14:26:34.944308+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:34.938407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:34.938407+0000","last_clean_scrub_stamp":"2023-10-06T14:26:34.938407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:48:11.028866+0000","scrub_duration":0.031095200999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":942,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:36.495949+0000","last_change":"2023-10-06T14:26:36.500436+0000","last_active":"2023-10-06T14:26:36.495949+0000","last_peered":"2023-10-06T14:26:36.495949+0000","last_clean":"2023-10-06T14:26:36.495949+0000","last_became_active":"2023-10-06T13:54:31.908571+0000","last_became_peered":"2023-10-06T13:54:31.908571+0000","last_unstale":"2023-10-06T14:26:36.495949+0000","last_undegraded":"2023-10-06T14:26:36.495949+0000","last_fullsized":"2023-10-06T14:26:36.495949+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:36.495123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:36.495123+0000","last_clean_scrub_stamp":"2023-10-06T14:26:36.495123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:21:09.895426+0000","scrub_duration":0.061798104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":944,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:27:12.402341+0000","last_change":"2023-10-06T14:27:12.402341+0000","last_active":"2023-10-06T14:27:12.402341+0000","last_peered":"2023-10-06T14:27:12.402341+0000","last_clean":"2023-10-06T14:27:12.402341+0000","last_became_active":"2023-10-06T14:22:39.078831+0000","last_became_peered":"2023-10-06T14:22:39.078831+0000","last_unstale":"2023-10-06T14:27:12.402341+0000","last_undegraded":"2023-10-06T14:27:12.402341+0000","last_fullsized":"2023-10-06T14:27:12.402341+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:27:12.399720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:27:12.399720+0000","last_clean_scrub_stamp":"2023-10-06T14:27:12.399720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:13.607206+0000","scrub_duration":0.030426088,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":933,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:39.351473+0000","last_change":"2023-10-06T14:26:39.353096+0000","last_active":"2023-10-06T14:26:39.351473+0000","last_peered":"2023-10-06T14:26:39.351473+0000","last_clean":"2023-10-06T14:26:39.351473+0000","last_became_active":"2023-10-06T14:05:13.230098+0000","last_became_peered":"2023-10-06T14:05:13.230098+0000","last_unstale":"2023-10-06T14:26:39.351473+0000","last_undegraded":"2023-10-06T14:26:39.351473+0000","last_fullsized":"2023-10-06T14:26:39.351473+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:39.350763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:39.350763+0000","last_clean_scrub_stamp":"2023-10-06T14:26:39.350763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:19:54.414820+0000","scrub_duration":0.064104507000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":942,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:39.196099+0000","last_change":"2023-10-06T14:26:39.200691+0000","last_active":"2023-10-06T14:26:39.196099+0000","last_peered":"2023-10-06T14:26:39.196099+0000","last_clean":"2023-10-06T14:26:39.196099+0000","last_became_active":"2023-10-06T14:16:33.635681+0000","last_became_peered":"2023-10-06T14:16:33.635681+0000","last_unstale":"2023-10-06T14:26:39.196099+0000","last_undegraded":"2023-10-06T14:26:39.196099+0000","last_fullsized":"2023-10-06T14:26:39.196099+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:39.195361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:39.195361+0000","last_clean_scrub_stamp":"2023-10-06T14:26:39.195361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:03:57.492939+0000","scrub_duration":0.040464247000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":942,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:41.020728+0000","last_change":"2023-10-06T14:26:41.022362+0000","last_active":"2023-10-06T14:26:41.020728+0000","last_peered":"2023-10-06T14:26:41.020728+0000","last_clean":"2023-10-06T14:26:41.020728+0000","last_became_active":"2023-10-06T14:16:33.269648+0000","last_became_peered":"2023-10-06T14:16:33.269648+0000","last_unstale":"2023-10-06T14:26:41.020728+0000","last_undegraded":"2023-10-06T14:26:41.020728+0000","last_fullsized":"2023-10-06T14:26:41.020728+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:41.020007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:41.020007+0000","last_clean_scrub_stamp":"2023-10-06T14:26:41.020007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:31.705678+0000","scrub_duration":0.032194962000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":954,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:27:08.419948+0000","last_change":"2023-10-06T14:27:08.425195+0000","last_active":"2023-10-06T14:27:08.419948+0000","last_peered":"2023-10-06T14:27:08.419948+0000","last_clean":"2023-10-06T14:27:08.419948+0000","last_became_active":"2023-10-06T14:22:39.121176+0000","last_became_peered":"2023-10-06T14:22:39.121176+0000","last_unstale":"2023-10-06T14:27:08.419948+0000","last_undegraded":"2023-10-06T14:27:08.419948+0000","last_fullsized":"2023-10-06T14:27:08.419948+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:27:08.419153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:27:08.419153+0000","last_clean_scrub_stamp":"2023-10-06T14:27:08.419153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:28:08.046830+0000","scrub_duration":0.032057210000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":937,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:34.528621+0000","last_change":"2023-10-06T14:26:34.528621+0000","last_active":"2023-10-06T14:26:34.528621+0000","last_peered":"2023-10-06T14:26:34.528621+0000","last_clean":"2023-10-06T14:26:34.528621+0000","last_became_active":"2023-10-06T13:54:27.642873+0000","last_became_peered":"2023-10-06T13:54:27.642873+0000","last_unstale":"2023-10-06T14:26:34.528621+0000","last_undegraded":"2023-10-06T14:26:34.528621+0000","last_fullsized":"2023-10-06T14:26:34.528621+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:34.522838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:34.522838+0000","last_clean_scrub_stamp":"2023-10-06T14:26:34.522838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:59:08.227916+0000","scrub_duration":0.081521365999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":941,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:58.469058+0000","last_change":"2023-10-06T14:26:58.469058+0000","last_active":"2023-10-06T14:26:58.469058+0000","last_peered":"2023-10-06T14:26:58.469058+0000","last_clean":"2023-10-06T14:26:58.469058+0000","last_became_active":"2023-10-06T14:22:39.733473+0000","last_became_peered":"2023-10-06T14:22:39.733473+0000","last_unstale":"2023-10-06T14:26:58.469058+0000","last_undegraded":"2023-10-06T14:26:58.469058+0000","last_fullsized":"2023-10-06T14:26:58.469058+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:58.461144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:58.461144+0000","last_clean_scrub_stamp":"2023-10-06T14:26:58.461144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:12.414312+0000","scrub_duration":0.034892911999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"970'78","reported_seq":1070,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:34.515865+0000","last_change":"2023-10-06T14:26:34.521747+0000","last_active":"2023-10-06T14:26:34.515865+0000","last_peered":"2023-10-06T14:26:34.515865+0000","last_clean":"2023-10-06T14:26:34.515865+0000","last_became_active":"2023-10-06T14:16:33.632776+0000","last_became_peered":"2023-10-06T14:16:33.632776+0000","last_unstale":"2023-10-06T14:26:34.515865+0000","last_undegraded":"2023-10-06T14:26:34.515865+0000","last_fullsized":"2023-10-06T14:26:34.515865+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"970'78","last_scrub_stamp":"2023-10-06T14:26:34.514526+0000","last_deep_scrub":"970'78","last_deep_scrub_stamp":"2023-10-06T14:26:34.514526+0000","last_clean_scrub_stamp":"2023-10-06T14:26:34.514526+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:00.523848+0000","scrub_duration":0.114787636,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":922,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:42.132958+0000","last_change":"2023-10-06T14:26:42.134644+0000","last_active":"2023-10-06T14:26:42.132958+0000","last_peered":"2023-10-06T14:26:42.132958+0000","last_clean":"2023-10-06T14:26:42.132958+0000","last_became_active":"2023-10-06T13:54:26.193354+0000","last_became_peered":"2023-10-06T13:54:26.193354+0000","last_unstale":"2023-10-06T14:26:42.132958+0000","last_undegraded":"2023-10-06T14:26:42.132958+0000","last_fullsized":"2023-10-06T14:26:42.132958+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:42.132239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:42.132239+0000","last_clean_scrub_stamp":"2023-10-06T14:26:42.132239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:55:22.212657+0000","scrub_duration":0.056253063999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":944,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:33.365482+0000","last_change":"2023-10-06T14:26:33.367228+0000","last_active":"2023-10-06T14:26:33.365482+0000","last_peered":"2023-10-06T14:26:33.365482+0000","last_clean":"2023-10-06T14:26:33.365482+0000","last_became_active":"2023-10-06T13:54:22.005243+0000","last_became_peered":"2023-10-06T13:54:22.005243+0000","last_unstale":"2023-10-06T14:26:33.365482+0000","last_undegraded":"2023-10-06T14:26:33.365482+0000","last_fullsized":"2023-10-06T14:26:33.365482+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:33.364656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:33.364656+0000","last_clean_scrub_stamp":"2023-10-06T14:26:33.364656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:23:09.595356+0000","scrub_duration":0.043849388000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"970'8","reported_seq":952,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:37.931903+0000","last_change":"2023-10-06T14:26:37.936968+0000","last_active":"2023-10-06T14:26:37.931903+0000","last_peered":"2023-10-06T14:26:37.931903+0000","last_clean":"2023-10-06T14:26:37.931903+0000","last_became_active":"2023-10-06T14:05:13.458219+0000","last_became_peered":"2023-10-06T14:05:13.458219+0000","last_unstale":"2023-10-06T14:26:37.931903+0000","last_undegraded":"2023-10-06T14:26:37.931903+0000","last_fullsized":"2023-10-06T14:26:37.931903+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"970'8","last_scrub_stamp":"2023-10-06T14:26:37.931101+0000","last_deep_scrub":"970'8","last_deep_scrub_stamp":"2023-10-06T14:26:37.931101+0000","last_clean_scrub_stamp":"2023-10-06T14:26:37.931101+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:29.815005+0000","scrub_duration":0.09831413,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":944,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:43.006334+0000","last_change":"2023-10-06T14:26:43.007863+0000","last_active":"2023-10-06T14:26:43.006334+0000","last_peered":"2023-10-06T14:26:43.006334+0000","last_clean":"2023-10-06T14:26:43.006334+0000","last_became_active":"2023-10-06T14:22:39.269435+0000","last_became_peered":"2023-10-06T14:22:39.269435+0000","last_unstale":"2023-10-06T14:26:43.006334+0000","last_undegraded":"2023-10-06T14:26:43.006334+0000","last_fullsized":"2023-10-06T14:26:43.006334+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":835,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:43.005623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:43.005623+0000","last_clean_scrub_stamp":"2023-10-06T14:26:43.005623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:06:44.934607+0000","scrub_duration":0.033589740999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":935,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:45.113928+0000","last_change":"2023-10-06T14:26:45.113928+0000","last_active":"2023-10-06T14:26:45.113928+0000","last_peered":"2023-10-06T14:26:45.113928+0000","last_clean":"2023-10-06T14:26:45.113928+0000","last_became_active":"2023-10-06T14:16:33.650127+0000","last_became_peered":"2023-10-06T14:16:33.650127+0000","last_unstale":"2023-10-06T14:26:45.113928+0000","last_undegraded":"2023-10-06T14:26:45.113928+0000","last_fullsized":"2023-10-06T14:26:45.113928+0000","mapping_epoch":659,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":660,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:45.111370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:45.111370+0000","last_clean_scrub_stamp":"2023-10-06T14:26:45.111370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:38.367962+0000","scrub_duration":0.062333412999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":924,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:46.863778+0000","last_change":"2023-10-06T14:26:46.863778+0000","last_active":"2023-10-06T14:26:46.863778+0000","last_peered":"2023-10-06T14:26:46.863778+0000","last_clean":"2023-10-06T14:26:46.863778+0000","last_became_active":"2023-10-06T13:54:22.160421+0000","last_became_peered":"2023-10-06T13:54:22.160421+0000","last_unstale":"2023-10-06T14:26:46.863778+0000","last_undegraded":"2023-10-06T14:26:46.863778+0000","last_fullsized":"2023-10-06T14:26:46.863778+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:46.861353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:46.861353+0000","last_clean_scrub_stamp":"2023-10-06T14:26:46.861353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:52:25.904881+0000","scrub_duration":0.052787556999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":943,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:35.440991+0000","last_change":"2023-10-06T14:26:35.442788+0000","last_active":"2023-10-06T14:26:35.440991+0000","last_peered":"2023-10-06T14:26:35.440991+0000","last_clean":"2023-10-06T14:26:35.440991+0000","last_became_active":"2023-10-06T13:54:25.349894+0000","last_became_peered":"2023-10-06T13:54:25.349894+0000","last_unstale":"2023-10-06T14:26:35.440991+0000","last_undegraded":"2023-10-06T14:26:35.440991+0000","last_fullsized":"2023-10-06T14:26:35.440991+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:35.440184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:35.440184+0000","last_clean_scrub_stamp":"2023-10-06T14:26:35.440184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:26.848807+0000","scrub_duration":0.052068601999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":957,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:34.317404+0000","last_change":"2023-10-06T14:26:34.319556+0000","last_active":"2023-10-06T14:26:34.317404+0000","last_peered":"2023-10-06T14:26:34.317404+0000","last_clean":"2023-10-06T14:26:34.317404+0000","last_became_active":"2023-10-06T14:22:39.979167+0000","last_became_peered":"2023-10-06T14:22:39.979167+0000","last_unstale":"2023-10-06T14:26:34.317404+0000","last_undegraded":"2023-10-06T14:26:34.317404+0000","last_fullsized":"2023-10-06T14:26:34.317404+0000","mapping_epoch":834,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":836,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:34.316567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:34.316567+0000","last_clean_scrub_stamp":"2023-10-06T14:26:34.316567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:33:39.123933+0000","scrub_duration":0.035959370999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":954,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:31.449380+0000","last_change":"2023-10-06T14:26:31.449380+0000","last_active":"2023-10-06T14:26:31.449380+0000","last_peered":"2023-10-06T14:26:31.449380+0000","last_clean":"2023-10-06T14:26:31.449380+0000","last_became_active":"2023-10-06T14:05:13.191757+0000","last_became_peered":"2023-10-06T14:05:13.191757+0000","last_unstale":"2023-10-06T14:26:31.449380+0000","last_undegraded":"2023-10-06T14:26:31.449380+0000","last_fullsized":"2023-10-06T14:26:31.449380+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:31.445079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:31.445079+0000","last_clean_scrub_stamp":"2023-10-06T14:26:31.445079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:17:11.350420+0000","scrub_duration":0.083960115000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":923,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:36.882247+0000","last_change":"2023-10-06T14:26:36.882247+0000","last_active":"2023-10-06T14:26:36.882247+0000","last_peered":"2023-10-06T14:26:36.882247+0000","last_clean":"2023-10-06T14:26:36.882247+0000","last_became_active":"2023-10-06T13:54:22.144576+0000","last_became_peered":"2023-10-06T13:54:22.144576+0000","last_unstale":"2023-10-06T14:26:36.882247+0000","last_undegraded":"2023-10-06T14:26:36.882247+0000","last_fullsized":"2023-10-06T14:26:36.882247+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:36.878675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:36.878675+0000","last_clean_scrub_stamp":"2023-10-06T14:26:36.878675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:18:58.847490+0000","scrub_duration":0.050518194000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":941,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:46.697951+0000","last_change":"2023-10-06T14:26:46.699751+0000","last_active":"2023-10-06T14:26:46.697951+0000","last_peered":"2023-10-06T14:26:46.697951+0000","last_clean":"2023-10-06T14:26:46.697951+0000","last_became_active":"2023-10-06T13:54:19.734381+0000","last_became_peered":"2023-10-06T13:54:19.734381+0000","last_unstale":"2023-10-06T14:26:46.697951+0000","last_undegraded":"2023-10-06T14:26:46.697951+0000","last_fullsized":"2023-10-06T14:26:46.697951+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:46.697067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:46.697067+0000","last_clean_scrub_stamp":"2023-10-06T14:26:46.697067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:16:20.063672+0000","scrub_duration":0.065551264999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":948,"reported_epoch":970,"state":"active+clean","last_fresh":"2023-10-06T14:26:30.418675+0000","last_change":"2023-10-06T14:26:30.426357+0000","last_active":"2023-10-06T14:26:30.418675+0000","last_peered":"2023-10-06T14:26:30.418675+0000","last_clean":"2023-10-06T14:26:30.418675+0000","last_became_active":"2023-10-06T14:05:13.442612+0000","last_became_peered":"2023-10-06T14:05:13.442612+0000","last_unstale":"2023-10-06T14:26:30.418675+0000","last_undegraded":"2023-10-06T14:26:30.418675+0000","last_fullsized":"2023-10-06T14:26:30.418675+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T14:26:30.417803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T14:26:30.417803+0000","last_clean_scrub_stamp":"2023-10-06T14:26:30.417803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:23:39.298614+0000","scrub_duration":0.036831065000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":16,"seq":68719477132,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59132,"kb_used_data":5940,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94312708,"statfs":{"total":96636764160,"available":96576212992,"internally_reserved":0,"allocated":6082560,"data_stored":3417737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":11.946,"5min":12.488,"15min":13.279},"min":{"1min":6.3920000000000003,"5min":4.6070000000000002,"15min":4.6070000000000002},"max":{"1min":52.079999999999998,"5min":91.744,"15min":109.691},"last":6.3529999999999998},{"interface":"front","average":{"1min":12.875999999999999,"5min":12.664,"15min":14.576000000000001},"min":{"1min":4.1870000000000003,"5min":4.1870000000000003,"15min":4.1870000000000003},"max":{"1min":44.232999999999997,"5min":67.936000000000007,"15min":166.33699999999999},"last":12.16}]},{"osd":2,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":12.112,"5min":11.595000000000001,"15min":13.598000000000001},"min":{"1min":7.9850000000000003,"5min":6.3920000000000003,"15min":4.9800000000000004},"max":{"1min":18.631,"5min":64.661000000000001,"15min":155.68799999999999},"last":7.2969999999999997},{"interface":"front","average":{"1min":15.757999999999999,"5min":13.82,"15min":16.335000000000001},"min":{"1min":9.0660000000000007,"5min":7.3049999999999997,"15min":6.6059999999999999},"max":{"1min":47.368000000000002,"5min":92.335999999999999,"15min":212.16399999999999},"last":13.298999999999999}]},{"osd":3,"last update":"Fri Oct 6 14:26:49 2023","interfaces":[{"interface":"back","average":{"1min":12.4,"5min":25.032,"15min":32.039999999999999},"min":{"1min":8.2949999999999999,"5min":8.2949999999999999,"15min":8.2949999999999999},"max":{"1min":19.728999999999999,"5min":310.19,"15min":310.19},"last":11.573},{"interface":"front","average":{"1min":12.678000000000001,"5min":17.033999999999999,"15min":18.442},"min":{"1min":7.5030000000000001,"5min":7.5030000000000001,"15min":7.5030000000000001},"max":{"1min":18.052,"5min":65.375,"15min":65.375},"last":10.182}]},{"osd":4,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":15290.522000000001,"5min":3070.721,"15min":1038.3879999999999},"min":{"1min":9.2720000000000002,"5min":7.8520000000000003,"15min":7.0190000000000001},"max":{"1min":39693.332999999999,"5min":39693.332999999999,"15min":39693.332999999999},"last":15.791},{"interface":"front","average":{"1min":15323.151,"5min":3078.1880000000001,"15min":1039.8889999999999},"min":{"1min":14.579000000000001,"5min":8.5679999999999996,"15min":7.3239999999999998},"max":{"1min":39697.402000000002,"5min":39697.402000000002,"15min":39697.402000000002},"last":10.84}]},{"osd":5,"last update":"Fri Oct 6 14:26:49 2023","interfaces":[{"interface":"back","average":{"1min":14.340999999999999,"5min":16.838000000000001,"15min":90.614999999999995},"min":{"1min":9.3989999999999991,"5min":9.1199999999999992,"15min":9.1199999999999992},"max":{"1min":19.346,"5min":69.561999999999998,"15min":8890.4069999999992},"last":9.5719999999999992},{"interface":"front","average":{"1min":12.846,"5min":16.812000000000001,"15min":90.933999999999997},"min":{"1min":7.3239999999999998,"5min":7.3239999999999998,"15min":7.3239999999999998},"max":{"1min":21.536000000000001,"5min":139.78800000000001,"15min":8888.7939999999999},"last":8.1980000000000004}]},{"osd":6,"last update":"Fri Oct 6 14:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.563000000000001,"5min":15.667999999999999,"15min":1233.97},"min":{"1min":9.9359999999999999,"5min":8.4710000000000001,"15min":7.3529999999999998},"max":{"1min":20.318000000000001,"5min":78.221999999999994,"15min":47304.226999999999},"last":12.734},{"interface":"front","average":{"1min":17.091000000000001,"5min":18.989000000000001,"15min":1237.328},"min":{"1min":11.202999999999999,"5min":10.666,"15min":8.9949999999999992},"max":{"1min":24.579000000000001,"5min":116.84999999999999,"15min":47302.955999999998},"last":16.646000000000001}]},{"osd":7,"last update":"Fri Oct 6 14:26:33 2023","interfaces":[{"interface":"back","average":{"1min":17.274999999999999,"5min":20.236000000000001,"15min":22.026},"min":{"1min":11.840999999999999,"5min":10.992000000000001,"15min":9.0250000000000004},"max":{"1min":23.812000000000001,"5min":78.769000000000005,"15min":172.16300000000001},"last":17.506},{"interface":"front","average":{"1min":15.042999999999999,"5min":20.027999999999999,"15min":21.492000000000001},"min":{"1min":9.7680000000000007,"5min":9.7680000000000007,"15min":9.7560000000000002},"max":{"1min":30.442,"5min":134.17400000000001,"15min":180.739},"last":15.042999999999999}]}]},{"osd":0,"up_from":16,"seq":68719477148,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58740,"kb_used_data":5548,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313100,"statfs":{"total":96636764160,"available":96576614400,"internally_reserved":0,"allocated":5681152,"data_stored":3019897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 14:26:15 2023","interfaces":[{"interface":"back","average":{"1min":19.928999999999998,"5min":17.440999999999999,"15min":16.654},"min":{"1min":6.7800000000000002,"5min":5.4980000000000002,"15min":4.9720000000000004},"max":{"1min":97.855999999999995,"5min":135.13800000000001,"15min":342.61399999999998},"last":6.7279999999999998},{"interface":"front","average":{"1min":21.222000000000001,"5min":19.111999999999998,"15min":17.628},"min":{"1min":5.9409999999999998,"5min":5.8810000000000002,"15min":5.5960000000000001},"max":{"1min":144.221,"5min":144.221,"15min":337.48000000000002},"last":13.952}]},{"osd":2,"last update":"Fri Oct 6 14:26:25 2023","interfaces":[{"interface":"back","average":{"1min":24.416,"5min":21.916,"15min":19.238},"min":{"1min":5.5030000000000001,"5min":5.5030000000000001,"15min":5.5030000000000001},"max":{"1min":231.68000000000001,"5min":231.68000000000001,"15min":231.68000000000001},"last":13.34},{"interface":"front","average":{"1min":20.329000000000001,"5min":17.545000000000002,"15min":17.381},"min":{"1min":6.6470000000000002,"5min":5.718,"15min":5.718},"max":{"1min":121.98399999999999,"5min":181.06700000000001,"15min":181.06700000000001},"last":7.9429999999999996}]},{"osd":3,"last update":"Fri Oct 6 14:26:46 2023","interfaces":[{"interface":"back","average":{"1min":14.472,"5min":22.420999999999999,"15min":24.835000000000001},"min":{"1min":9.1199999999999992,"5min":7.4139999999999997,"15min":7.4139999999999997},"max":{"1min":38.506999999999998,"5min":189.10499999999999,"15min":189.10499999999999},"last":16.102},{"interface":"front","average":{"1min":18.135000000000002,"5min":23.192,"15min":24.527000000000001},"min":{"1min":11.356999999999999,"5min":6.9850000000000003,"15min":6.9850000000000003},"max":{"1min":46.523000000000003,"5min":163.60900000000001,"15min":163.60900000000001},"last":18.824000000000002}]},{"osd":4,"last update":"Fri Oct 6 14:27:07 2023","interfaces":[{"interface":"back","average":{"1min":6896.491,"5min":2123.3310000000001,"15min":721.23800000000006},"min":{"1min":7.3840000000000003,"5min":7.3840000000000003,"15min":6.9269999999999996},"max":{"1min":35207.186999999998,"5min":39913.453999999998,"15min":39913.453999999998},"last":11.534000000000001},{"interface":"front","average":{"1min":6893.0379999999996,"5min":2118.2429999999999,"15min":718.92499999999995},"min":{"1min":7.2709999999999999,"5min":7.2050000000000001,"15min":7.2050000000000001},"max":{"1min":35205.514999999999,"5min":39912.421999999999,"15min":39912.421999999999},"last":10.311999999999999}]},{"osd":5,"last update":"Fri Oct 6 14:26:50 2023","interfaces":[{"interface":"back","average":{"1min":25.170999999999999,"5min":18.625,"15min":70.325999999999993},"min":{"1min":11.279,"5min":7.5209999999999999,"15min":7.5209999999999999},"max":{"1min":172.108,"5min":172.108,"15min":6380.991},"last":17.908999999999999},{"interface":"front","average":{"1min":19.899999999999999,"5min":21.440000000000001,"15min":65.507000000000005},"min":{"1min":8.5370000000000008,"5min":8.5370000000000008,"15min":8.5370000000000008},"max":{"1min":87.198999999999998,"5min":145.59299999999999,"15min":6266.0159999999996},"last":9.3070000000000004}]},{"osd":6,"last update":"Fri Oct 6 14:27:05 2023","interfaces":[{"interface":"back","average":{"1min":19.706,"5min":21.186,"15min":584.95000000000005},"min":{"1min":7.8040000000000003,"5min":7.1479999999999997,"15min":7.1479999999999997},"max":{"1min":99.099999999999994,"5min":177.73699999999999,"15min":42325.411999999997},"last":12.709},{"interface":"front","average":{"1min":22.815000000000001,"5min":24.405000000000001,"15min":583.63999999999999},"min":{"1min":12.997,"5min":8.6609999999999996,"15min":6.9459999999999997},"max":{"1min":91.759,"5min":181.625,"15min":42298.872000000003},"last":19.457999999999998}]},{"osd":7,"last update":"Fri Oct 6 14:26:33 2023","interfaces":[{"interface":"back","average":{"1min":23.925000000000001,"5min":24.867999999999999,"15min":1322.9580000000001},"min":{"1min":10.121,"5min":10.121,"15min":9.8209999999999997},"max":{"1min":96.147000000000006,"5min":178.86500000000001,"15min":67028.107999999993},"last":16.998999999999999},{"interface":"front","average":{"1min":19.460000000000001,"5min":20.870999999999999,"15min":1320.289},"min":{"1min":7.8730000000000002,"5min":7.8730000000000002,"15min":7.8730000000000002},"max":{"1min":89.558999999999997,"5min":178.38900000000001,"15min":67001.737999999998},"last":14.675000000000001}]}]},{"osd":4,"up_from":430,"seq":1846835937665,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58744,"kb_used_data":5552,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313096,"statfs":{"total":96636764160,"available":96576610304,"internally_reserved":0,"allocated":5685248,"data_stored":3019902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:27:09 2023","interfaces":[{"interface":"back","average":{"1min":9.7349999999999994,"5min":21.32,"15min":15.361000000000001},"min":{"1min":4.6159999999999997,"5min":4.508,"15min":4.085},"max":{"1min":23.364000000000001,"5min":469.096,"15min":469.096},"last":7.3780000000000001},{"interface":"front","average":{"1min":12.576000000000001,"5min":20.977,"15min":17.027999999999999},"min":{"1min":6.4390000000000001,"5min":4.758,"15min":4.5999999999999996},"max":{"1min":45.811999999999998,"5min":423.42000000000002,"15min":423.42000000000002},"last":9.2089999999999996}]},{"osd":1,"last update":"Fri Oct 6 14:26:13 2023","interfaces":[{"interface":"back","average":{"1min":81.888000000000005,"5min":27.257000000000001,"15min":19.550000000000001},"min":{"1min":7.3289999999999997,"5min":5.9850000000000003,"15min":4.9390000000000001},"max":{"1min":469.72800000000001,"5min":469.72800000000001,"15min":469.72800000000001},"last":8.6910000000000007},{"interface":"front","average":{"1min":77.108999999999995,"5min":26.469999999999999,"15min":21.765999999999998},"min":{"1min":5.9320000000000004,"5min":5.9320000000000004,"15min":5.9320000000000004},"max":{"1min":425.584,"5min":425.584,"15min":425.584},"last":10.926}]},{"osd":2,"last update":"Fri Oct 6 14:27:09 2023","interfaces":[{"interface":"back","average":{"1min":12.147,"5min":34.731000000000002,"15min":23.68},"min":{"1min":6.5110000000000001,"5min":6.5110000000000001,"15min":6.4770000000000003},"max":{"1min":42.548000000000002,"5min":530.30399999999997,"15min":530.30399999999997},"last":6.7640000000000002},{"interface":"front","average":{"1min":15.613,"5min":37.847000000000001,"15min":26.661000000000001},"min":{"1min":9.2439999999999998,"5min":7.6239999999999997,"15min":6.5780000000000003},"max":{"1min":51.658000000000001,"5min":530.86099999999999,"15min":530.86099999999999},"last":14.721}]},{"osd":3,"last update":"Fri Oct 6 14:27:09 2023","interfaces":[{"interface":"back","average":{"1min":12.718999999999999,"5min":24.552,"15min":19.004000000000001},"min":{"1min":7.1509999999999998,"5min":7.1509999999999998,"15min":7.1509999999999998},"max":{"1min":30.591999999999999,"5min":470.80099999999999,"15min":470.80099999999999},"last":16.548999999999999},{"interface":"front","average":{"1min":15.625999999999999,"5min":22.326000000000001,"15min":17.187000000000001},"min":{"1min":7.1689999999999996,"5min":6.7489999999999997,"15min":6.7489999999999997},"max":{"1min":52.576999999999998,"5min":422.88,"15min":422.88},"last":15.412000000000001}]},{"osd":5,"last update":"Fri Oct 6 14:27:09 2023","interfaces":[{"interface":"back","average":{"1min":15.994999999999999,"5min":24.260000000000002,"15min":76.751000000000005},"min":{"1min":7.9500000000000002,"5min":7.9500000000000002,"15min":5.9509999999999996},"max":{"1min":53.106999999999999,"5min":431.57900000000001,"15min":8813.2510000000002},"last":9.8940000000000001},{"interface":"front","average":{"1min":14.692,"5min":23.277999999999999,"15min":77.682000000000002},"min":{"1min":8.3330000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"max":{"1min":39.529000000000003,"5min":537.68399999999997,"15min":8811.7579999999998},"last":12.231999999999999}]},{"osd":6,"last update":"Fri Oct 6 14:26:13 2023","interfaces":[{"interface":"back","average":{"1min":96.150999999999996,"5min":31.527000000000001,"15min":920.85000000000002},"min":{"1min":9.7840000000000007,"5min":6.9470000000000001,"15min":6.9470000000000001},"max":{"1min":563.85900000000004,"5min":563.85900000000004,"15min":44817.595000000001},"last":12.941000000000001},{"interface":"front","average":{"1min":84.043000000000006,"5min":30.527999999999999,"15min":922.12},"min":{"1min":12.715,"5min":7.5830000000000002,"15min":7.3789999999999996},"max":{"1min":471.80700000000002,"5min":471.80700000000002,"15min":44816.752999999997},"last":17.393000000000001}]},{"osd":7,"last update":"Fri Oct 6 14:27:09 2023","interfaces":[{"interface":"back","average":{"1min":18.91,"5min":23.863,"15min":1630.194},"min":{"1min":13.708,"5min":9.7119999999999997,"15min":7.8310000000000004},"max":{"1min":44.176000000000002,"5min":564.822,"15min":65382.432000000001},"last":14.083},{"interface":"front","average":{"1min":18.283000000000001,"5min":22.405999999999999,"15min":1626.9860000000001},"min":{"1min":11.627000000000001,"5min":7.8639999999999999,"15min":7.4059999999999997},"max":{"1min":61.856000000000002,"5min":433.30799999999999,"15min":65371.873},"last":11.648999999999999}]}]},{"osd":3,"up_from":834,"seq":3582002725227,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58740,"kb_used_data":5548,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313100,"statfs":{"total":96636764160,"available":96576614400,"internally_reserved":0,"allocated":5681152,"data_stored":3019897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:40 2023","interfaces":[{"interface":"back","average":{"1min":11.488,"5min":22.166,"15min":30.718},"min":{"1min":5.7679999999999998,"5min":4.8559999999999999,"15min":4.8559999999999999},"max":{"1min":46.276000000000003,"5min":188.56700000000001,"15min":188.56700000000001},"last":17.100000000000001},{"interface":"front","average":{"1min":13.634,"5min":23.224,"15min":30.103000000000002},"min":{"1min":6.4980000000000002,"5min":5.827,"15min":5.827},"max":{"1min":49.347000000000001,"5min":214.209,"15min":214.209},"last":10.862}]},{"osd":1,"last update":"Fri Oct 6 14:26:40 2023","interfaces":[{"interface":"back","average":{"1min":20.433,"5min":34.404000000000003,"15min":33.829999999999998},"min":{"1min":7.7089999999999996,"5min":7.4409999999999998,"15min":7.4409999999999998},"max":{"1min":122.386,"5min":993.702,"15min":993.702},"last":11.971},{"interface":"front","average":{"1min":22.824000000000002,"5min":36.021000000000001,"15min":35.810000000000002},"min":{"1min":5.8869999999999996,"5min":5.8869999999999996,"15min":5.8869999999999996},"max":{"1min":147.75800000000001,"5min":991.50599999999997,"15min":991.50599999999997},"last":17.794}]},{"osd":2,"last update":"Fri Oct 6 14:26:40 2023","interfaces":[{"interface":"back","average":{"1min":15.666,"5min":25.468,"15min":32.722999999999999},"min":{"1min":8.0190000000000001,"5min":8.0190000000000001,"15min":8.0190000000000001},"max":{"1min":59.93,"5min":230.25399999999999,"15min":230.25399999999999},"last":16.738},{"interface":"front","average":{"1min":15.298999999999999,"5min":23.792000000000002,"15min":29.678000000000001},"min":{"1min":8.3569999999999993,"5min":8.1959999999999997,"15min":8.1959999999999997},"max":{"1min":58.402000000000001,"5min":165.553,"15min":165.553},"last":12.683999999999999}]},{"osd":4,"last update":"Fri Oct 6 14:27:10 2023","interfaces":[{"interface":"back","average":{"1min":8778.0439999999999,"5min":2321.6419999999998,"15min":798.87400000000002},"min":{"1min":8.9649999999999999,"5min":7.1020000000000003,"15min":7.1020000000000003},"max":{"1min":36735.343999999997,"5min":40828.069000000003,"15min":40828.069000000003},"last":19.561},{"interface":"front","average":{"1min":8861.3909999999996,"5min":1794.076,"15min":621.05799999999999},"min":{"1min":7.3129999999999997,"5min":7.3129999999999997,"15min":7.3129999999999997},"max":{"1min":36927.300999999999,"5min":36927.300999999999,"15min":36927.300999999999},"last":14.436}]},{"osd":5,"last update":"Fri Oct 6 14:26:40 2023","interfaces":[{"interface":"back","average":{"1min":18.367000000000001,"5min":26.66,"15min":33.622},"min":{"1min":9.7880000000000003,"5min":9.7880000000000003,"15min":9.7880000000000003},"max":{"1min":78.909000000000006,"5min":196.50200000000001,"15min":196.50200000000001},"last":15.859999999999999},{"interface":"front","average":{"1min":16.731000000000002,"5min":23.861000000000001,"15min":29.673999999999999},"min":{"1min":8.2129999999999992,"5min":7.9329999999999998,"15min":7.9329999999999998},"max":{"1min":60.256,"5min":177.739,"15min":177.739},"last":20.100999999999999}]},{"osd":6,"last update":"Fri Oct 6 14:26:40 2023","interfaces":[{"interface":"back","average":{"1min":24.187999999999999,"5min":33.337000000000003,"15min":45.566000000000003},"min":{"1min":9.1679999999999993,"5min":9.1679999999999993,"15min":9.1679999999999993},"max":{"1min":127.839,"5min":381.13,"15min":381.13},"last":20.763999999999999},{"interface":"front","average":{"1min":24.706,"5min":29.417999999999999,"15min":36.106999999999999},"min":{"1min":9.0510000000000002,"5min":9.0510000000000002,"15min":9.0510000000000002},"max":{"1min":123.29600000000001,"5min":245.16200000000001,"15min":245.16200000000001},"last":11.555999999999999}]},{"osd":7,"last update":"Fri Oct 6 14:26:40 2023","interfaces":[{"interface":"back","average":{"1min":22.215,"5min":25.847999999999999,"15min":28.803999999999998},"min":{"1min":9.8019999999999996,"5min":9.8019999999999996,"15min":9.8019999999999996},"max":{"1min":95.766999999999996,"5min":227.42599999999999,"15min":227.42599999999999},"last":15.189},{"interface":"front","average":{"1min":20.015999999999998,"5min":23.420999999999999,"15min":24.927},"min":{"1min":10.595000000000001,"5min":10.595000000000001,"15min":10.595000000000001},"max":{"1min":86.596999999999994,"5min":164.09899999999999,"15min":164.09899999999999},"last":21.913}]}]},{"osd":2,"up_from":15,"seq":64424509842,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58740,"kb_used_data":5548,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94313100,"statfs":{"total":96636764160,"available":96576614400,"internally_reserved":0,"allocated":5681152,"data_stored":3019897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:27:09 2023","interfaces":[{"interface":"back","average":{"1min":13.176,"5min":12.314,"15min":15.500999999999999},"min":{"1min":6.6180000000000003,"5min":6.0069999999999997,"15min":5.3630000000000004},"max":{"1min":37.688000000000002,"5min":37.688000000000002,"15min":182.96600000000001},"last":7.5460000000000003},{"interface":"front","average":{"1min":12.308,"5min":12.028,"15min":16.954999999999998},"min":{"1min":6.0309999999999997,"5min":4.6260000000000003,"15min":4.6260000000000003},"max":{"1min":32.590000000000003,"5min":44.423000000000002,"15min":226.68600000000001},"last":8.9960000000000004}]},{"osd":1,"last update":"Fri Oct 6 14:27:09 2023","interfaces":[{"interface":"back","average":{"1min":15.436999999999999,"5min":16.806999999999999,"15min":17.632999999999999},"min":{"1min":6.6449999999999996,"5min":6.6449999999999996,"15min":6.0970000000000004},"max":{"1min":38.289000000000001,"5min":101.874,"15min":148.20400000000001},"last":9.6969999999999992},{"interface":"front","average":{"1min":14.055,"5min":15.210000000000001,"15min":17.059000000000001},"min":{"1min":6.2549999999999999,"5min":6.2549999999999999,"15min":6.2549999999999999},"max":{"1min":51.273000000000003,"5min":161.97499999999999,"15min":229.82499999999999},"last":17.276}]},{"osd":3,"last update":"Fri Oct 6 14:26:50 2023","interfaces":[{"interface":"back","average":{"1min":18.111999999999998,"5min":17.122,"15min":16.521999999999998},"min":{"1min":8.1780000000000008,"5min":8.1780000000000008,"15min":8.1780000000000008},"max":{"1min":62.646999999999998,"5min":62.646999999999998,"15min":62.646999999999998},"last":12.414},{"interface":"front","average":{"1min":17.864000000000001,"5min":15.199,"15min":14.981},"min":{"1min":8.2080000000000002,"5min":8.2080000000000002,"15min":8.2080000000000002},"max":{"1min":64.054000000000002,"5min":64.054000000000002,"15min":64.054000000000002},"last":18.102}]},{"osd":4,"last update":"Fri Oct 6 14:27:09 2023","interfaces":[{"interface":"back","average":{"1min":17.167000000000002,"5min":2991.4110000000001,"15min":1009.389},"min":{"1min":9.2859999999999996,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":70.638000000000005,"5min":38405.120000000003,"15min":38405.120000000003},"last":18.908999999999999},{"interface":"front","average":{"1min":13.855,"5min":2970.5169999999998,"15min":1005.153},"min":{"1min":7.5359999999999996,"5min":7.5359999999999996,"15min":7.5359999999999996},"max":{"1min":28.445,"5min":38264.116000000002,"15min":38264.116000000002},"last":11.16}]},{"osd":5,"last update":"Fri Oct 6 14:26:52 2023","interfaces":[{"interface":"back","average":{"1min":19.434999999999999,"5min":17.443000000000001,"15min":90.012},"min":{"1min":10.007999999999999,"5min":8.2560000000000002,"15min":7.6589999999999998},"max":{"1min":73.159000000000006,"5min":73.159000000000006,"15min":9721.0490000000009},"last":19.709},{"interface":"front","average":{"1min":16.032,"5min":16.637,"15min":94.126000000000005},"min":{"1min":7.3090000000000002,"5min":7.3090000000000002,"15min":7.3090000000000002},"max":{"1min":77.673000000000002,"5min":77.673000000000002,"15min":9793.7890000000007},"last":13.757999999999999}]},{"osd":6,"last update":"Fri Oct 6 14:27:09 2023","interfaces":[{"interface":"back","average":{"1min":17.693999999999999,"5min":17.888000000000002,"15min":22.670000000000002},"min":{"1min":8.6820000000000004,"5min":8.6820000000000004,"15min":8.6820000000000004},"max":{"1min":34.725999999999999,"5min":44.606999999999999,"15min":231.273},"last":14.696},{"interface":"front","average":{"1min":18.516999999999999,"5min":17.878,"15min":20.890000000000001},"min":{"1min":8.6180000000000003,"5min":7.5330000000000004,"15min":7.141},"max":{"1min":76.936000000000007,"5min":76.936000000000007,"15min":179.232},"last":11.792}]},{"osd":7,"last update":"Fri Oct 6 14:26:36 2023","interfaces":[{"interface":"back","average":{"1min":20.045999999999999,"5min":20.690000000000001,"15min":1733.6900000000001},"min":{"1min":9.5600000000000005,"5min":9.5600000000000005,"15min":8.6189999999999998},"max":{"1min":73.903000000000006,"5min":73.903000000000006,"15min":69773.953999999998},"last":15.353999999999999},{"interface":"front","average":{"1min":20.099,"5min":19.152999999999999,"15min":1723.25},"min":{"1min":10.614000000000001,"5min":9.5909999999999993,"15min":8.5429999999999993},"max":{"1min":78.433999999999997,"5min":78.433999999999997,"15min":69547.547999999995},"last":13.019}]}]},{"osd":6,"up_from":14,"seq":60129542541,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62840,"kb_used_data":5552,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94309000,"statfs":{"total":96636764160,"available":96572416000,"internally_reserved":0,"allocated":5685248,"data_stored":3019902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:27:15 2023","interfaces":[{"interface":"back","average":{"1min":8.4260000000000002,"5min":11.036,"15min":11.506},"min":{"1min":4.8470000000000004,"5min":4.8470000000000004,"15min":4.4050000000000002},"max":{"1min":14.238,"5min":94.040999999999997,"15min":173.703},"last":9.7520000000000007},{"interface":"front","average":{"1min":11.855,"5min":15.661,"15min":15.417},"min":{"1min":7.6779999999999999,"5min":7.5309999999999997,"15min":4.5140000000000002},"max":{"1min":16.702000000000002,"5min":130.036,"15min":147.483},"last":12.117000000000001}]},{"osd":1,"last update":"Fri Oct 6 14:27:15 2023","interfaces":[{"interface":"back","average":{"1min":9.9819999999999993,"5min":26.831,"15min":19.707000000000001},"min":{"1min":6.6639999999999997,"5min":5.4059999999999997,"15min":5.4059999999999997},"max":{"1min":14.087,"5min":1271.6179999999999,"15min":1271.6179999999999},"last":10.839},{"interface":"front","average":{"1min":13.101000000000001,"5min":30.870000000000001,"15min":23.364000000000001},"min":{"1min":8.3450000000000006,"5min":6.907,"15min":6.8639999999999999},"max":{"1min":18.584,"5min":1322.71,"15min":1322.71},"last":15.512}]},{"osd":2,"last update":"Fri Oct 6 14:27:15 2023","interfaces":[{"interface":"back","average":{"1min":14.253,"5min":15.529,"15min":15.228999999999999},"min":{"1min":8.6980000000000004,"5min":6.1660000000000004,"15min":5.532},"max":{"1min":30.056999999999999,"5min":101.197,"15min":221.06800000000001},"last":14.800000000000001},{"interface":"front","average":{"1min":10.747999999999999,"5min":17.033999999999999,"15min":19.440999999999999},"min":{"1min":6.9450000000000003,"5min":6.4509999999999996,"15min":6.4509999999999996},"max":{"1min":22.808,"5min":99.492999999999995,"15min":234.125},"last":6.9450000000000003}]},{"osd":3,"last update":"Fri Oct 6 14:26:43 2023","interfaces":[{"interface":"back","average":{"1min":17.905999999999999,"5min":19.57,"15min":19.963000000000001},"min":{"1min":9.3699999999999992,"5min":6.8470000000000004,"15min":6.8470000000000004},"max":{"1min":75.385999999999996,"5min":126.55500000000001,"15min":126.55500000000001},"last":12.833},{"interface":"front","average":{"1min":11.311,"5min":18.966999999999999,"15min":22.920000000000002},"min":{"1min":7.6619999999999999,"5min":6.0389999999999997,"15min":6.0389999999999997},"max":{"1min":16.341000000000001,"5min":236.27000000000001,"15min":236.27000000000001},"last":14.025}]},{"osd":4,"last update":"Fri Oct 6 14:27:15 2023","interfaces":[{"interface":"back","average":{"1min":14.606999999999999,"5min":3179.828,"15min":1073.8499999999999},"min":{"1min":9.0020000000000007,"5min":9.0020000000000007,"15min":8.7490000000000006},"max":{"1min":28.978999999999999,"5min":38720.860999999997,"15min":38720.860999999997},"last":11.504},{"interface":"front","average":{"1min":15.972,"5min":3178.299,"15min":1071.021},"min":{"1min":7.9610000000000003,"5min":6.782,"15min":6.782},"max":{"1min":31.731000000000002,"5min":38719.173000000003,"15min":38719.173000000003},"last":17.306999999999999}]},{"osd":5,"last update":"Fri Oct 6 14:26:51 2023","interfaces":[{"interface":"back","average":{"1min":16.805,"5min":21.788,"15min":105.855},"min":{"1min":10.189,"5min":8.8800000000000008,"15min":8.8800000000000008},"max":{"1min":28.484000000000002,"5min":219.06200000000001,"15min":8032.4189999999999},"last":18.195},{"interface":"front","average":{"1min":17.248999999999999,"5min":21.498000000000001,"15min":108.09099999999999},"min":{"1min":11.912000000000001,"5min":7.1459999999999999,"15min":7.1459999999999999},"max":{"1min":29.532,"5min":171.041,"15min":8030.5749999999998},"last":13.377000000000001}]},{"osd":7,"last update":"Fri Oct 6 14:26:38 2023","interfaces":[{"interface":"back","average":{"1min":16.32,"5min":19.835000000000001,"15min":1130.444},"min":{"1min":13.643000000000001,"5min":11.01,"15min":11.01},"max":{"1min":19.600999999999999,"5min":93.355999999999995,"15min":69750.712},"last":19.175999999999998},{"interface":"front","average":{"1min":10.865,"5min":14.667,"15min":1124.432},"min":{"1min":6.806,"5min":6.4770000000000003,"15min":5.984},"max":{"1min":14.300000000000001,"5min":121.146,"15min":69749.794999999998},"last":8.6219999999999999}]}]},{"osd":5,"up_from":659,"seq":2830383448450,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77180,"kb_used_data":5940,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94294660,"statfs":{"total":96636764160,"available":96557731840,"internally_reserved":0,"allocated":6082560,"data_stored":3417737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:54 2023","interfaces":[{"interface":"back","average":{"1min":13.567,"5min":14.255000000000001,"15min":35.558999999999997},"min":{"1min":4.4870000000000001,"5min":4.4870000000000001,"15min":4.1879999999999997},"max":{"1min":62.234999999999999,"5min":173.54599999999999,"15min":680.28200000000004},"last":10.369999999999999},{"interface":"front","average":{"1min":21.788,"5min":14.276999999999999,"15min":27.896999999999998},"min":{"1min":4.7869999999999999,"5min":4.633,"15min":4.5579999999999998},"max":{"1min":157.27600000000001,"5min":157.27600000000001,"15min":608.46199999999999},"last":7.431}]},{"osd":1,"last update":"Fri Oct 6 14:26:54 2023","interfaces":[{"interface":"back","average":{"1min":12.805999999999999,"5min":20.707000000000001,"15min":39.976999999999997},"min":{"1min":5.1539999999999999,"5min":5.1539999999999999,"15min":5.1539999999999999},"max":{"1min":70.140000000000001,"5min":633.78700000000003,"15min":633.78700000000003},"last":9.7829999999999995},{"interface":"front","average":{"1min":15.073,"5min":24.794,"15min":42.438000000000002},"min":{"1min":5.9630000000000001,"5min":5.9000000000000004,"15min":5.8259999999999996},"max":{"1min":81.863,"5min":598.60000000000002,"15min":701.19399999999996},"last":10.667}]},{"osd":2,"last update":"Fri Oct 6 14:26:54 2023","interfaces":[{"interface":"back","average":{"1min":18.097000000000001,"5min":16.012,"15min":40.890000000000001},"min":{"1min":8.3960000000000008,"5min":6.8369999999999997,"15min":6.6029999999999998},"max":{"1min":114.88,"5min":114.88,"15min":603.024},"last":11.327999999999999},{"interface":"front","average":{"1min":18.84,"5min":18.355,"15min":45.579000000000001},"min":{"1min":7.8639999999999999,"5min":7.8639999999999999,"15min":7.8639999999999999},"max":{"1min":127.938,"5min":127.938,"15min":690.71799999999996},"last":12.468}]},{"osd":3,"last update":"Fri Oct 6 14:26:46 2023","interfaces":[{"interface":"back","average":{"1min":18.097000000000001,"5min":24.934999999999999,"15min":31.651},"min":{"1min":8.0079999999999991,"5min":8.0079999999999991,"15min":8.0079999999999991},"max":{"1min":76.787000000000006,"5min":264.37900000000002,"15min":264.37900000000002},"last":11.872999999999999},{"interface":"front","average":{"1min":18.696000000000002,"5min":24.536000000000001,"15min":29.300000000000001},"min":{"1min":9.5779999999999994,"5min":9.0990000000000002,"15min":9.0990000000000002},"max":{"1min":83.009,"5min":173.858,"15min":173.858},"last":16.398}]},{"osd":4,"last update":"Fri Oct 6 14:27:11 2023","interfaces":[{"interface":"back","average":{"1min":5973.3230000000003,"5min":1904.7429999999999,"15min":668.08600000000001},"min":{"1min":7.6440000000000001,"5min":7.6440000000000001,"15min":7.6440000000000001},"max":{"1min":35353.302000000003,"5min":38256.345000000001,"15min":38256.345000000001},"last":13.726000000000001},{"interface":"front","average":{"1min":5962.4740000000002,"5min":1903.6220000000001,"15min":673.95000000000005},"min":{"1min":7.2850000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"max":{"1min":35318.927000000003,"5min":38222.745999999999,"15min":38222.745999999999},"last":17.998999999999999}]},{"osd":6,"last update":"Fri Oct 6 14:26:54 2023","interfaces":[{"interface":"back","average":{"1min":19.481999999999999,"5min":19.565999999999999,"15min":43.625999999999998},"min":{"1min":10.701000000000001,"5min":7.7300000000000004,"15min":7.7300000000000004},"max":{"1min":80.486000000000004,"5min":108.053,"15min":607.87400000000002},"last":13.196},{"interface":"front","average":{"1min":21.062999999999999,"5min":22.984999999999999,"15min":47.561},"min":{"1min":10.032999999999999,"5min":10.032999999999999,"15min":7.1040000000000001},"max":{"1min":115.67700000000001,"5min":115.67700000000001,"15min":739.33100000000002},"last":17.199000000000002}]},{"osd":7,"last update":"Fri Oct 6 14:26:54 2023","interfaces":[{"interface":"back","average":{"1min":21.905999999999999,"5min":22.684999999999999,"15min":44.311},"min":{"1min":10.147,"5min":9.3569999999999993,"15min":7.0350000000000001},"max":{"1min":118.07299999999999,"5min":118.07299999999999,"15min":609.59799999999996},"last":18.768000000000001},{"interface":"front","average":{"1min":15.606999999999999,"5min":14.206,"15min":40.584000000000003},"min":{"1min":6.6260000000000003,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":125.985,"5min":125.985,"15min":610.43799999999999},"last":15.476000000000001}]}]},{"osd":7,"up_from":13,"seq":55834575243,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80884,"kb_used_data":5548,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94290956,"statfs":{"total":96636764160,"available":96553938944,"internally_reserved":0,"allocated":5681152,"data_stored":3019897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 14:26:31 2023","interfaces":[{"interface":"back","average":{"1min":8.9030000000000005,"5min":10.318,"15min":32.875999999999998},"min":{"1min":4.8620000000000001,"5min":4.742,"15min":4.2539999999999996},"max":{"1min":23.603000000000002,"5min":46.529000000000003,"15min":930.62099999999998},"last":16.815999999999999},{"interface":"front","average":{"1min":11.708,"5min":11.972,"15min":31.646999999999998},"min":{"1min":5.1619999999999999,"5min":4.8079999999999998,"15min":4.8079999999999998},"max":{"1min":39.262,"5min":69.477999999999994,"15min":879.35199999999998},"last":7.2569999999999997}]},{"osd":1,"last update":"Fri Oct 6 14:26:31 2023","interfaces":[{"interface":"back","average":{"1min":11.952999999999999,"5min":26.451000000000001,"15min":39.228000000000002},"min":{"1min":6.9059999999999997,"5min":5.4790000000000001,"15min":5.4790000000000001},"max":{"1min":23.271000000000001,"5min":1094.5889999999999,"15min":1094.5889999999999},"last":8.6839999999999993},{"interface":"front","average":{"1min":12.124000000000001,"5min":25.449000000000002,"15min":24.937000000000001},"min":{"1min":6.2110000000000003,"5min":6.2110000000000003,"15min":5.4539999999999997},"max":{"1min":28.507999999999999,"5min":1057.856,"15min":1057.856},"last":13.605}]},{"osd":2,"last update":"Fri Oct 6 14:26:31 2023","interfaces":[{"interface":"back","average":{"1min":11.727,"5min":15.356,"15min":23.838000000000001},"min":{"1min":6.9969999999999999,"5min":6.5149999999999997,"15min":6.202},"max":{"1min":30.532,"5min":75.823999999999998,"15min":316.84399999999999},"last":11.003},{"interface":"front","average":{"1min":14.474,"5min":16.344000000000001,"15min":39.460000000000001},"min":{"1min":8.4879999999999995,"5min":5.6500000000000004,"15min":5.6500000000000004},"max":{"1min":31.562999999999999,"5min":98.183000000000007,"15min":931.94500000000005},"last":9.0210000000000008}]},{"osd":3,"last update":"Fri Oct 6 14:26:47 2023","interfaces":[{"interface":"back","average":{"1min":11.794,"5min":18.402999999999999,"15min":22.134},"min":{"1min":7.6349999999999998,"5min":6.9969999999999999,"15min":6.9969999999999999},"max":{"1min":16.472999999999999,"5min":184.93700000000001,"15min":184.93700000000001},"last":12.521000000000001},{"interface":"front","average":{"1min":14.417999999999999,"5min":17.114000000000001,"15min":18.498999999999999},"min":{"1min":7.54,"5min":7.54,"15min":7.54},"max":{"1min":19.154,"5min":77.144000000000005,"15min":77.144000000000005},"last":15.456}]},{"osd":4,"last update":"Fri Oct 6 14:27:08 2023","interfaces":[{"interface":"back","average":{"1min":6545.0870000000004,"5min":1746.115,"15min":602.81700000000001},"min":{"1min":8.0359999999999996,"5min":5.9429999999999996,"15min":5.9429999999999996},"max":{"1min":37300.904999999999,"5min":40210.642,"15min":40210.642},"last":9.8390000000000004},{"interface":"front","average":{"1min":6610.1729999999998,"5min":1762.088,"15min":603.05600000000004},"min":{"1min":11.365,"5min":7.4180000000000001,"15min":7.0010000000000003},"max":{"1min":37468.290000000001,"5min":40377.220000000001,"15min":40377.220000000001},"last":14.269}]},{"osd":5,"last update":"Fri Oct 6 14:26:50 2023","interfaces":[{"interface":"back","average":{"1min":12.994999999999999,"5min":18.428000000000001,"15min":50.965000000000003},"min":{"1min":6.9790000000000001,"5min":6.9790000000000001,"15min":6.9790000000000001},"max":{"1min":17.867000000000001,"5min":149.61799999999999,"15min":6750.3770000000004},"last":16.233000000000001},{"interface":"front","average":{"1min":13.414,"5min":19.297999999999998,"15min":51.856000000000002},"min":{"1min":8.0630000000000006,"5min":7.4009999999999998,"15min":6.9290000000000003},"max":{"1min":19.074000000000002,"5min":174.839,"15min":6875.5780000000004},"last":11.629}]},{"osd":6,"last update":"Fri Oct 6 14:26:31 2023","interfaces":[{"interface":"back","average":{"1min":15.701000000000001,"5min":18.128,"15min":26.704000000000001},"min":{"1min":10.023,"5min":7.9509999999999996,"15min":7.7290000000000001},"max":{"1min":36.390000000000001,"5min":75.013999999999996,"15min":380.74000000000001},"last":18.317},{"interface":"front","average":{"1min":18.978000000000002,"5min":20.349,"15min":44.131},"min":{"1min":9.2360000000000007,"5min":8.9489999999999998,"15min":8.9489999999999998},"max":{"1min":82.266999999999996,"5min":120.83799999999999,"15min":935.49300000000005},"last":13.074}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T14:27:18.811 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T14:27:18.916 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.914+0000 7f66c5c39640 10 monclient: get_monmap_and_config 2023-10-06T14:27:18.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.914+0000 7f66c5c39640 10 monclient: build_initial_monmap 2023-10-06T14:27:18.917 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.914+0000 7f66c5c39640 10 monclient: monmap: 2023-10-06T14:27:18.917 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:27:18.917 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:27:18.917 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T14:27:18.915657+0000 2023-10-06T14:27:18.917 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T14:27:18.915657+0000 2023-10-06T14:27:18.917 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:27:18.917 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:27:18.917 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-10-06T14:27:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.83:6789/0 mon.noname-c-legacy 2023-10-06T14:27:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-10-06T14:27:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:3: v2:172.21.15.19:3300/0 mon.noname-a 2023-10-06T14:27:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:4: v2:172.21.15.83:3300/0 mon.noname-c 2023-10-06T14:27:18.918 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:27:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.915+0000 7f66c5c39640 1 Processor -- start 2023-10-06T14:27:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.915+0000 7f66c5c39640 1 -- start start 2023-10-06T14:27:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.915+0000 7f66c5c39640 10 monclient: init 2023-10-06T14:27:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.915+0000 7f66c5c39640 10 monclient: _reopen_session rank -1 2023-10-06T14:27:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.915+0000 7f66c5c39640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2023-10-06T14:27:18.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.915+0000 7f66c5c39640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f66c00b09b0 0x7f66c00aee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:27:18.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.915+0000 7f66c5c39640 10 monclient(hunting): picked mon.noname-c con 0x7f66c00b09b0 addr v2:172.21.15.83:3300/0 2023-10-06T14:27:18.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.915+0000 7f66c5c39640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f66c00af390 0x7f66c0065920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:27:18.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.915+0000 7f66c5c39640 10 monclient(hunting): picked mon.noname-b con 0x7f66c00af390 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:27:18.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66c5c39640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f66c0065e60 addr v1:172.21.15.83:6789/0 2023-10-06T14:27:18.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66c5c39640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c00b0d80 con 0x7f66c0065e60 2023-10-06T14:27:18.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66c5c39640 10 monclient(hunting): start opening mon connection 2023-10-06T14:27:18.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66c5c39640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c00b0010 con 0x7f66c00b09b0 2023-10-06T14:27:18.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66c5c39640 10 monclient(hunting): start opening mon connection 2023-10-06T14:27:18.921 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66c5c39640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c00af7c0 con 0x7f66c00af390 2023-10-06T14:27:18.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66c5c39640 10 monclient(hunting): _renew_subs 2023-10-06T14:27:18.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66c5c39640 10 monclient(hunting): authenticate will time out at 3367.102408s 2023-10-06T14:27:18.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66beffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f66c00af390 0x7f66c0065920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:27:18.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66bf7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f66c00b09b0 0x7f66c00aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:27:18.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66beffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f66c00af390 0x7f66c0065920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:55240/0 (socket says 172.21.15.19:55240) 2023-10-06T14:27:18.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66beffd640 1 -- 172.21.15.19:0/160913025 learned_addr learned my addr 172.21.15.19:0/160913025 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:27:18.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66beffd640 10 monclient(hunting): get_auth_request con 0x7f66c00af390 auth_method 0 2023-10-06T14:27:18.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:27:18.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66beffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:27:18.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66beffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:27:18.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66bf7fe640 10 monclient(hunting): get_auth_request con 0x7f66c00b09b0 auth_method 0 2023-10-06T14:27:18.922 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:27:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66bf7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:27:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.916+0000 7f66bf7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:27:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.917+0000 7f66beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:27:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.917+0000 7f66beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:27:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.917+0000 7f66beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:27:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.917+0000 7f66bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:27:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.917+0000 7f66bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:27:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.917+0000 7f66bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:27:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.919+0000 7f66beffd640 10 monclient(hunting): handle_auth_done global_id 17228 payload 274 2023-10-06T14:27:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.919+0000 7f66beffd640 1 -- 172.21.15.19:0/160913025 >> v2:172.21.15.83:3300/0 conn(0x7f66c00b09b0 msgr2=0x7f66c00aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T14:27:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.919+0000 7f66beffd640 1 --2- 172.21.15.19:0/160913025 >> v2:172.21.15.83:3300/0 conn(0x7f66c00b09b0 0x7f66c00aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.919+0000 7f66beffd640 1 -- 172.21.15.19:0/160913025 >> v1:172.21.15.83:6789/0 conn(0x7f66c0065e60 legacy=0x7f66c0140d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:27:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.919+0000 7f66beffd640 10 monclient: _finish_hunting 0 2023-10-06T14:27:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.919+0000 7f66beffd640 1 monclient: found mon.noname-b 2023-10-06T14:27:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.919+0000 7f66beffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:27:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.919+0000 7f66beffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-10-06T14:27:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.919+0000 7f66beffd640 1 -- 172.21.15.19:0/160913025 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c00a8c60 con 0x7f66c00af390 2023-10-06T14:27:18.925 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.920+0000 7f66beffd640 1 --2- 172.21.15.19:0/160913025 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f66c00af390 0x7f66c0065920 secure :-1 s=READY pgs=3134 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009870 tx=0x7f66b402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9725a8a6f6329bdb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:27:18.931 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66be7fc640 1 -- 172.21.15.19:0/160913025 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4030840 con 0x7f66c00af390 2023-10-06T14:27:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:27:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66be7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:27:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66be7fc640 10 monclient: dump: 2023-10-06T14:27:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:27:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:27:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:27:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:27:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:27:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:27:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:27:18.934 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:27:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66be7fc640 10 monclient: _finish_auth 0 2023-10-06T14:27:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66be7fc640 10 monclient: _check_auth_tickets 2023-10-06T14:27:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:27:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66be7fc640 1 -- 172.21.15.19:0/160913025 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b40309a0 con 0x7f66c00af390 2023-10-06T14:27:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66be7fc640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:27:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66be7fc640 1 -- 172.21.15.19:0/160913025 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4030d30 con 0x7f66c00af390 2023-10-06T14:27:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66c5c39640 5 monclient: authenticate success, global_id 17228 2023-10-06T14:27:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66c5c39640 10 monclient: get_monmap_and_config success 2023-10-06T14:27:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66c5c39640 4 set_mon_vals no callback set 2023-10-06T14:27:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:27:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66be7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-10-06T14:27:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.924+0000 7f66be7fc640 10 monclient: dump: 2023-10-06T14:27:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:27:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:27:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:27:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:27:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:27:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:27:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:27:18.936 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:27:18.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.925+0000 7f66c5c39640 10 monclient: shutdown 2023-10-06T14:27:18.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.925+0000 7f66c5c39640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:27:18.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.925+0000 7f66c5c39640 1 -- 172.21.15.19:0/160913025 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f66c00af390 msgr2=0x7f66c0065920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:27:18.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.925+0000 7f66c5c39640 1 --2- 172.21.15.19:0/160913025 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f66c00af390 0x7f66c0065920 secure :-1 s=READY pgs=3134 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009870 tx=0x7f66b402b980 comp rx=0 tx=0).stop 2023-10-06T14:27:18.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.925+0000 7f66c5c39640 1 -- 172.21.15.19:0/160913025 shutdown_connections 2023-10-06T14:27:18.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.925+0000 7f66c5c39640 1 --2- 172.21.15.19:0/160913025 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f66c00af390 0x7f66c0065920 unknown :-1 s=CLOSED pgs=3134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.925+0000 7f66c5c39640 1 --2- 172.21.15.19:0/160913025 >> v2:172.21.15.83:3300/0 conn(0x7f66c00b09b0 0x7f66c00aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.925+0000 7f66c5c39640 1 -- 172.21.15.19:0/160913025 >> 172.21.15.19:0/160913025 conn(0x7f66c0069c00 msgr2=0x7f66c0069fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:27:18.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.925+0000 7f66c5c39640 1 -- 172.21.15.19:0/160913025 shutdown_connections 2023-10-06T14:27:18.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.925+0000 7f66c5c39640 1 -- 172.21.15.19:0/160913025 wait complete. 2023-10-06T14:27:18.936 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.926+0000 7f66c5c39640 10 monclient: build_initial_monmap 2023-10-06T14:27:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.926+0000 7f66c5c39640 10 monclient: monmap: 2023-10-06T14:27:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-10-06T14:27:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-10-06T14:27:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-10-06T14:27:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-10-06T14:27:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-10-06T14:27:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:27:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-10-06T14:27:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.noname-b 2023-10-06T14:27:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-c 2023-10-06T14:27:18.937 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:27:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.926+0000 7f66c5c39640 1 librados: starting msgr at 2023-10-06T14:27:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.926+0000 7f66c5c39640 1 librados: starting objecter 2023-10-06T14:27:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.926+0000 7f66c5c39640 1 Processor -- start 2023-10-06T14:27:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.926+0000 7f66c5c39640 1 -- start start 2023-10-06T14:27:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.926+0000 7f66c5c39640 1 librados: setting wanted keys 2023-10-06T14:27:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.926+0000 7f66c5c39640 1 librados: calling monclient init 2023-10-06T14:27:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.926+0000 7f66c5c39640 10 monclient: init 2023-10-06T14:27:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 10 monclient: _reopen_session rank -1 2023-10-06T14:27:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 10 monclient: _add_conns ranks=[0,2,1] 2023-10-06T14:27:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c0065e60 0x7f66c013e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:27:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 10 monclient(hunting): picked mon.noname-a con 0x7f66c0065e60 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-10-06T14:27:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f66c00af390 0x7f66c00b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:27:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 10 monclient(hunting): picked mon.noname-c con 0x7f66c00af390 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-10-06T14:27:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c00b09b0 0x7f66c01d8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:27:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 10 monclient(hunting): picked mon.noname-b con 0x7f66c00b09b0 addr [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2023-10-06T14:27:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 10 monclient(hunting): start opening mon connection 2023-10-06T14:27:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c01c7fb0 con 0x7f66c0065e60 2023-10-06T14:27:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 10 monclient(hunting): start opening mon connection 2023-10-06T14:27:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c0070380 con 0x7f66c00b09b0 2023-10-06T14:27:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 10 monclient(hunting): start opening mon connection 2023-10-06T14:27:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c0070660 con 0x7f66c00af390 2023-10-06T14:27:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 10 monclient(hunting): _renew_subs 2023-10-06T14:27:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66c5c39640 10 monclient(hunting): authenticate will time out at 3367.113782s 2023-10-06T14:27:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66beffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f66c00af390 0x7f66c00b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:27:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.927+0000 7f66bffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c00b09b0 0x7f66c01d8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:27:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66beffd640 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f66c00af390 0x7f66c00b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:55244/0 (socket says 172.21.15.19:55244) 2023-10-06T14:27:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66beffd640 1 -- 172.21.15.19:0/3709457701 learned_addr learned my addr 172.21.15.19:0/3709457701 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-10-06T14:27:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66beffd640 10 monclient(hunting): get_auth_request con 0x7f66c00af390 auth_method 0 2023-10-06T14:27:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:27:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66beffd640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:27:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66beffd640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:27:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66bf7fe640 1 --2- 172.21.15.19:0/3709457701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c0065e60 0x7f66c013e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:27:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66bffff640 10 monclient(hunting): get_auth_request con 0x7f66c00b09b0 auth_method 0 2023-10-06T14:27:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:27:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66bffff640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:27:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66bffff640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:27:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66bf7fe640 10 monclient(hunting): get_auth_request con 0x7f66c0065e60 auth_method 0 2023-10-06T14:27:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-10-06T14:27:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66bf7fe640 10 monclient(hunting): _init_auth method 2 2023-10-06T14:27:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66bf7fe640 10 monclient(hunting): _init_auth creating new auth 2023-10-06T14:27:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:27:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:27:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:27:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:27:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:27:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.928+0000 7f66bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:27:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.929+0000 7f66bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-10-06T14:27:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.929+0000 7f66bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-10-06T14:27:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.929+0000 7f66bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-10-06T14:27:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.931+0000 7f66bffff640 10 monclient(hunting): handle_auth_done global_id 20842 payload 824 2023-10-06T14:27:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.931+0000 7f66bffff640 1 -- 172.21.15.19:0/3709457701 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f66c00af390 msgr2=0x7f66c00b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:27:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.931+0000 7f66bffff640 1 --2- 172.21.15.19:0/3709457701 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f66c00af390 0x7f66c00b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.931+0000 7f66bffff640 1 -- 172.21.15.19:0/3709457701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c0065e60 msgr2=0x7f66c013e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:27:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.931+0000 7f66bffff640 1 --2- 172.21.15.19:0/3709457701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c0065e60 0x7f66c013e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.931+0000 7f66bffff640 10 monclient: _finish_hunting 0 2023-10-06T14:27:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.931+0000 7f66bffff640 1 monclient: found mon.noname-b 2023-10-06T14:27:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.931+0000 7f66bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-10-06T14:27:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.931+0000 7f66bffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.83:3300/0 2023-10-06T14:27:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.931+0000 7f66bffff640 1 -- 172.21.15.19:0/3709457701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b40094d0 con 0x7f66c00b09b0 2023-10-06T14:27:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.933+0000 7f66bffff640 1 --2- 172.21.15.19:0/3709457701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c00b09b0 0x7f66c01d8ea0 secure :-1 s=READY pgs=5142 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c310 tx=0x7f66b800c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:27:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.933+0000 7f66a7fff640 10 client.?.objecter ms_handle_connect 0x7f66c00b09b0 2023-10-06T14:27:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.933+0000 7f66a7fff640 10 client.?.objecter resend_mon_ops 2023-10-06T14:27:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66a7fff640 1 -- 172.21.15.19:0/3709457701 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b80052d0 con 0x7f66c00b09b0 2023-10-06T14:27:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:27:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66a7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-10-06T14:27:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66a7fff640 10 monclient: dump: 2023-10-06T14:27:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:27:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:27:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:27:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:27:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:27:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:27:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:27:18.945 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:27:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66a7fff640 10 monclient: _finish_auth 0 2023-10-06T14:27:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66a7fff640 10 monclient: _check_auth_tickets 2023-10-06T14:27:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-10-06T14:27:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66a7fff640 1 -- 172.21.15.19:0/3709457701 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b8005430 con 0x7f66c00b09b0 2023-10-06T14:27:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66c5c39640 5 monclient: authenticate success, global_id 20842 2023-10-06T14:27:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66c5c39640 10 monclient: _renew_subs 2023-10-06T14:27:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66c5c39640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:27:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66c5c39640 1 -- 172.21.15.19:0/3709457701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c0070b80 con 0x7f66c00b09b0 2023-10-06T14:27:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66a7fff640 10 monclient: handle_config config(0 keys) v1 2023-10-06T14:27:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66a7fff640 1 -- 172.21.15.19:0/3709457701 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8003830 con 0x7f66c00b09b0 2023-10-06T14:27:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-10-06T14:27:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66a7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-10-06T14:27:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66a7fff640 10 monclient: dump: 2023-10-06T14:27:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-10-06T14:27:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:fsid c5a90f7c-81ec-43a3-ab47-af6ce5729ec5 2023-10-06T14:27:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-10-06T13:51:23.675166+0000 2023-10-06T14:27:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-10-06T14:27:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-10-06T14:27:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-10-06T14:27:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2023-10-06T14:27:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2023-10-06T14:27:18.947 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-10-06T14:27:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66c5c39640 10 client.20842.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-10-06T14:27:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66c5c39640 10 monclient: _renew_subs 2023-10-06T14:27:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66c5c39640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:27:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66c5c39640 1 -- 172.21.15.19:0/3709457701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c01c8c40 con 0x7f66c00b09b0 2023-10-06T14:27:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66c5c39640 1 librados: init done 2023-10-06T14:27:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.937+0000 7f66bdffb640 4 set_mon_vals no callback set 2023-10-06T14:27:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.938+0000 7f66c5c39640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-10-06T14:27:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.938+0000 7f66c5c39640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:27:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.938+0000 7f66c5c39640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:27:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:18.938+0000 7f66c5c39640 1 -- 172.21.15.19:0/3709457701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c004e000 con 0x7f66c00b09b0 2023-10-06T14:27:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.176+0000 7f66a7fff640 1 -- 172.21.15.19:0/3709457701 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f66b8003990 con 0x7f66c00b09b0 2023-10-06T14:27:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.177+0000 7f66a7fff640 1 --2- 172.21.15.19:0/3709457701 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f66ac06c210 0x7f66ac06e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T14:27:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.177+0000 7f66bf7fe640 1 --2- 172.21.15.19:0/3709457701 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f66ac06c210 0x7f66ac06e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T14:27:19.329 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.177+0000 7f66bf7fe640 10 monclient: get_auth_request con 0x7f66ac06c210 auth_method 0 2023-10-06T14:27:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.178+0000 7f66bf7fe640 1 --2- 172.21.15.19:0/3709457701 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f66ac06c210 0x7f66ac06e6c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f66c00b3820 tx=0x7f66b000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T14:27:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.178+0000 7f66a7fff640 10 client.20842.objecter ms_handle_connect 0x7f66ac06c210 2023-10-06T14:27:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.179+0000 7f66a7fff640 1 -- 172.21.15.19:0/3709457701 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(970..970 src has 714..970) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f66b8057940 con 0x7f66c00b09b0 2023-10-06T14:27:19.340 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.179+0000 7f66a7fff640 10 client.20842.objecter ms_dispatch 0x7f66c0000b90 osd_map(970..970 src has 714..970) v4 2023-10-06T14:27:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.179+0000 7f66a7fff640 3 client.20842.objecter handle_osd_map got epochs [970,970] > 0 2023-10-06T14:27:19.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.179+0000 7f66a7fff640 3 client.20842.objecter handle_osd_map decoding full epoch 970 2023-10-06T14:27:19.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.179+0000 7f66a7fff640 20 client.20842.objecter dump_active .. 0 homeless 2023-10-06T14:27:19.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.189+0000 7f66a7fff640 1 -- 172.21.15.19:0/3709457701 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179012 (secure 0 0 0) 0x7f66b8053c80 con 0x7f66c00b09b0 2023-10-06T14:27:19.342 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.189+0000 7f66a7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-10-06T14:27:19.343 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.189+0000 7f66a7fff640 10 monclient: _finish_command 1 = system:0 2023-10-06T14:27:19.392 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.390+0000 7f66c5c39640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2023-10-06T14:27:19.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.390+0000 7f66c5c39640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2023-10-06T14:27:19.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.390+0000 7f66c5c39640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.83:3300/0 2023-10-06T14:27:19.393 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.390+0000 7f66c5c39640 1 -- 172.21.15.19:0/3709457701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f66c00704c0 con 0x7f66c00b09b0 2023-10-06T14:27:19.615 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.613+0000 7f66a7fff640 1 -- 172.21.15.19:0/3709457701 <== mon.1 v2:172.21.15.83:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f66b805b6c0 con 0x7f66c00b09b0 2023-10-06T14:27:19.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.613+0000 7f66a7fff640 10 monclient: handle_config config(1 keys) v1 2023-10-06T14:27:19.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.613+0000 7f66bdffb640 4 set_mon_vals no callback set 2023-10-06T14:27:19.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.613+0000 7f66bdffb640 10 set_mon_vals mon_health_to_clog = false 2023-10-06T14:27:19.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.613+0000 7f66a7fff640 1 -- 172.21.15.19:0/3709457701 <== mon.1 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) v1 ==== 123+0+0 (secure 0 0 0) 0x7f66c00704c0 con 0x7f66c00b09b0 2023-10-06T14:27:19.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.613+0000 7f66a7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2023-10-06T14:27:19.616 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.613+0000 7f66a7fff640 10 monclient: _finish_command 2 = system:0 2023-10-06T14:27:19.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.615+0000 7f66c5c39640 10 librados: watch_flush enter 2023-10-06T14:27:19.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.615+0000 7f66c5c39640 10 librados: watch_flush exit 2023-10-06T14:27:19.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.615+0000 7f66c5c39640 20 client.20842.objecter shutdown clearing up homeless session... 2023-10-06T14:27:19.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.615+0000 7f66c5c39640 10 client.20842.objecter successfully canceled tick 2023-10-06T14:27:19.618 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.615+0000 7f66c5c39640 1 -- 172.21.15.19:0/3709457701 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f66ac06c210 msgr2=0x7f66ac06e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:27:19.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.615+0000 7f66c5c39640 1 --2- 172.21.15.19:0/3709457701 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f66ac06c210 0x7f66ac06e6c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f66c00b3820 tx=0x7f66b000b040 comp rx=0 tx=0).stop 2023-10-06T14:27:19.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.615+0000 7f66c5c39640 10 monclient: shutdown 2023-10-06T14:27:19.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.615+0000 7f66c5c39640 20 monclient: shutdown discarding 0 pending message(s) 2023-10-06T14:27:19.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.615+0000 7f66c5c39640 1 -- 172.21.15.19:0/3709457701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c00b09b0 msgr2=0x7f66c01d8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T14:27:19.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.615+0000 7f66c5c39640 1 --2- 172.21.15.19:0/3709457701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c00b09b0 0x7f66c01d8ea0 secure :-1 s=READY pgs=5142 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c310 tx=0x7f66b800c7d0 comp rx=0 tx=0).stop 2023-10-06T14:27:19.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.615+0000 7f66c5c39640 1 -- 172.21.15.19:0/3709457701 shutdown_connections 2023-10-06T14:27:19.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.615+0000 7f66c5c39640 1 --2- 172.21.15.19:0/3709457701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c00b09b0 0x7f66c01d8ea0 unknown :-1 s=CLOSED pgs=5142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:19.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.615+0000 7f66c5c39640 1 --2- 172.21.15.19:0/3709457701 >> [v2:172.21.15.83:6800/85581,v1:172.21.15.83:6801/85581] conn(0x7f66ac06c210 0x7f66ac06e6c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:19.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.616+0000 7f66c5c39640 1 --2- 172.21.15.19:0/3709457701 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f66c00af390 0x7f66c00b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:19.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.616+0000 7f66c5c39640 1 --2- 172.21.15.19:0/3709457701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c0065e60 0x7f66c013e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T14:27:19.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.616+0000 7f66c5c39640 1 -- 172.21.15.19:0/3709457701 >> 172.21.15.19:0/3709457701 conn(0x7f66c0069c00 msgr2=0x7f66c00aab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T14:27:19.619 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.616+0000 7f66c5c39640 1 -- 172.21.15.19:0/3709457701 shutdown_connections 2023-10-06T14:27:19.620 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.616+0000 7f66c5c39640 1 -- 172.21.15.19:0/3709457701 wait complete. 2023-10-06T14:27:19.620 INFO:teuthology.orchestra.run.smithi019.stderr:2023-10-06T14:27:19.616+0000 7f66c5c39640 1 librados: shutdown 2023-10-06T14:27:19.630 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T14:27:19.630 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T14:27:19.630 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T14:27:19.630 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T14:27:19.646 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:27:19.633+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86178) UID: 0 2023-10-06T14:27:19.646 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-10-06T14:27:19.634+0000 b47e640 -1 osd.0 970 *** Got signal Terminated *** 2023-10-06T14:27:23.749 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:26.871 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:23.795 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:26.892 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:23.796 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:26.893 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:23.800 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:26.923 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:23.815 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:26.911 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:23.839 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:26.962 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:23.852 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:26.975 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:23.856 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:26.979 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:23.881 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:27.004 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:28.830 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:31.952 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:28.835 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:31.958 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:28.854 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:31.977 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:28.888 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:31.984 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:28.889 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:31.986 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:28.896 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:32.018 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:28.897 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:32.020 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:28.903 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:32.000 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:28.911 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:32.034 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:38.875 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:41.997 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:39.141 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:42.002 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:39.141 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:42.019 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:39.141 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:41.984 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:39.148 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:41.986 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:39.148 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:42.027 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:39.149 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:42.031 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:39.775 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:42.049 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:39.775 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:42.076 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:43.222 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-10-06T14:27:43.798 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:46.921 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:43.815 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:46.925 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:43.824 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:46.947 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:43.855 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:46.952 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:43.857 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:46.953 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:43.869 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:46.965 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:43.879 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:46.997 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:43.879 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:47.000 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:43.891 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:47.014 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:44.036 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T14:27:44.069 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T14:27:44.069 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T14:27:44.082 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:27:44.071+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86181) UID: 0 2023-10-06T14:27:44.097 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-10-06T14:27:44.072+0000 b47e640 -1 osd.1 972 *** Got signal Terminated *** 2023-10-06T14:27:44.309 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:47.405 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:44.309 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:47.406 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:44.435 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:35:47.415 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:44.435 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:47.509 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:44.435 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:47.510 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:44.436 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:35:47.519 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:44.436 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:47.532 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:44.436 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:47.533 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:44.436 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:35:47.542 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:49.331 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-10-06T14:27:49.559 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2023-10-06T14:27:50.171 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T14:27:50.319 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T14:27:50.319 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T14:27:50.338 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:27:50.323+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86185) UID: 0 2023-10-06T14:27:50.364 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-10-06T14:27:50.325+0000 b47e640 -1 osd.2 974 *** Got signal Terminated *** 2023-10-06T14:27:55.566 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-10-06T14:27:55.566 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-10-06T14:27:56.420 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T14:27:56.421 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T14:27:56.421 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T14:27:56.436 INFO:tasks.ceph.osd.3.smithi019.stderr:2023-10-06T14:27:56.424+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86187) UID: 0 2023-10-06T14:27:56.436 INFO:tasks.ceph.osd.3.smithi019.stderr:2023-10-06T14:27:56.425+0000 b47e640 -1 osd.3 978 *** Got signal Terminated *** 2023-10-06T14:27:58.845 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:01.967 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:58.848 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:01.971 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:58.870 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:01.992 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:58.892 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:01.989 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:58.895 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:01.991 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:58.897 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:02.020 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:58.900 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:02.023 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:58.914 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:02.011 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:27:58.921 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:02.043 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:01.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-10-06T14:28:01.484 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2023-10-06T14:28:02.522 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T14:28:02.522 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T14:28:02.522 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T14:28:02.535 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:28:02.525+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86115) UID: 0 2023-10-06T14:28:02.536 INFO:tasks.ceph.osd.4.smithi083.stderr:2023-10-06T14:28:02.526+0000 b47e640 -1 osd.4 980 *** Got signal Terminated *** 2023-10-06T14:28:04.032 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:07.155 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:04.036 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:07.159 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:04.049 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:07.172 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:04.082 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:07.179 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:04.083 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:07.180 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:04.095 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:07.191 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:04.097 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:07.220 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:04.100 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:07.223 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:04.111 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:07.234 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:07.290 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2023-10-06T14:28:07.314 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-10-06T14:28:08.624 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T14:28:08.624 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T14:28:08.624 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T14:28:08.637 INFO:tasks.ceph.osd.5.smithi083.stderr:2023-10-06T14:28:08.627+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86116) UID: 0 2023-10-06T14:28:08.637 INFO:tasks.ceph.osd.5.smithi083.stderr:2023-10-06T14:28:08.628+0000 b47e640 -1 osd.5 982 *** Got signal Terminated *** 2023-10-06T14:28:13.019 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-10-06T14:28:13.019 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2023-10-06T14:28:13.914 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:17.036 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:13.918 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:17.041 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:13.935 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:17.058 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:13.965 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:17.062 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:13.982 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:17.078 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:13.989 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:17.112 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:13.992 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:17.115 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:13.995 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:17.092 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:14.005 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:17.128 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:14.726 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T14:28:14.727 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T14:28:14.727 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T14:28:14.739 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:28:14.728+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86119) UID: 0 2023-10-06T14:28:14.742 INFO:tasks.ceph.osd.6.smithi083.stderr:2023-10-06T14:28:14.729+0000 b47e640 -1 osd.6 984 *** Got signal Terminated *** 2023-10-06T14:28:18.725 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-10-06T14:28:23.926 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:27.049 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:23.930 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:27.053 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:23.943 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:27.065 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:23.974 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:27.071 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:23.976 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:27.073 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:23.977 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:27.100 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:23.980 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:27.103 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:23.989 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:27.086 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:23.993 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:27.116 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:30.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-10-06T14:28:33.031 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T14:28:33.059 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T14:28:33.059 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T14:28:33.071 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:28:33.061+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86121) UID: 0 2023-10-06T14:28:33.104 INFO:tasks.ceph.osd.7.smithi083.stderr:2023-10-06T14:28:33.063+0000 b47e640 -1 osd.7 984 *** Got signal Terminated *** 2023-10-06T14:28:33.968 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:37.091 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:33.982 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:37.096 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:33.993 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:37.116 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:34.030 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:37.148 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:34.030 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:37.117 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:34.030 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:37.120 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:34.030 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:37.153 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:34.036 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:37.132 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:34.062 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:37.166 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:35.644 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-10-06T14:28:44.001 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:47.123 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:44.003 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:47.125 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:44.046 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:47.143 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:44.142 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:47.145 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:44.144 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:47.146 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:44.144 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:47.158 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:44.148 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:47.180 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:44.148 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:47.181 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:44.148 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:47.194 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:54.039 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:57.161 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:54.044 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:57.167 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:54.062 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:36:57.185 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:54.088 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:57.185 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:54.090 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:57.187 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:54.101 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:36:57.198 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:54.110 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:57.233 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:54.113 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:57.236 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:28:54.125 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:36:57.248 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:04.121 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:07.244 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:04.123 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:07.246 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:04.137 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:07.259 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:04.166 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:07.262 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:04.168 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:07.265 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:04.170 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:07.293 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:04.171 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:07.294 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:04.181 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:07.278 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:04.184 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:07.307 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:14.115 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:17.237 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:14.135 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:17.258 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:14.150 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:17.273 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:14.167 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:17.263 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:14.168 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:17.291 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:14.169 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:17.266 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:14.173 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:17.296 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:14.182 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:17.279 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:14.186 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:17.309 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:29.164 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:32.286 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:29.173 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:32.291 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:29.187 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:32.309 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:29.216 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:32.312 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:29.219 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:32.315 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:29.220 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:32.343 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:29.223 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:32.346 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:29.235 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:32.331 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:29.246 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:32.369 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:34.179 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:37.302 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:34.182 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:37.305 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:34.209 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:37.331 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:34.225 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:37.322 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:34.226 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:37.323 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:34.232 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:37.355 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:34.233 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:37.356 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:34.237 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:37.334 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:34.249 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:37.372 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:49.227 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:52.350 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:49.229 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:52.352 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:49.244 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:52.367 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:49.269 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:52.366 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:49.272 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:52.369 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:49.277 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:52.400 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:49.281 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:52.404 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:49.290 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:52.385 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:49.297 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:52.420 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:54.241 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:57.363 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:54.247 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:57.365 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:54.253 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:37:57.375 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:54.288 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:57.385 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:54.289 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:57.386 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:54.293 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:57.416 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:54.305 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:37:57.396 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:54.305 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:57.417 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:29:54.305 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:37:57.428 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:09.287 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:12.409 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:09.289 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:12.412 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:09.307 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:12.430 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:09.334 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:12.431 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:09.338 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:12.434 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:09.359 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:12.482 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:09.360 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:12.456 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:09.361 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:12.484 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:09.378 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:12.501 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:14.301 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:17.423 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:14.305 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:17.428 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:14.318 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:17.441 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:14.350 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:17.447 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:14.353 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:17.450 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:14.368 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:17.465 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:14.371 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:17.494 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:14.374 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:17.497 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:14.385 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:17.508 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:29.348 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:32.470 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:29.350 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:32.473 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:29.366 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:32.488 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:29.392 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:32.489 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:29.395 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:32.492 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:29.417 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:32.514 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:29.420 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:32.543 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:29.424 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:32.547 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:29.441 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:32.563 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:30.125 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:33.222 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:30.126 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:33.223 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:30.143 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:33.239 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:30.228 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:33.350 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:30.228 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:33.351 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:30.244 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:33.367 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:30.247 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:33.370 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:30.248 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:33.371 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:30.260 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:33.383 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:34.366 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:37.486 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:34.366 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:37.488 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:34.376 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:37.499 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:34.408 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:37.504 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:34.411 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:37.507 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:34.417 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:37.539 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:34.417 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:37.540 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:34.425 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:37.522 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:34.427 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:37.550 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:49.416 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:52.539 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:49.491 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:52.540 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:49.496 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:52.556 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:49.496 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:52.557 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:49.496 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:52.558 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:49.496 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:52.581 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:49.497 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:52.588 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:49.497 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:52.592 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:49.497 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:52.608 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:54.424 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:57.546 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:54.429 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:57.549 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:54.437 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:38:57.559 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:54.544 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:57.568 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:54.544 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:57.569 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:54.544 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:38:57.579 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:54.544 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:57.600 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:54.545 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:57.602 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:30:54.545 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:38:57.613 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:09.476 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:12.599 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:09.482 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:12.601 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:09.494 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:12.617 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:09.523 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:12.620 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:09.526 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:12.622 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:09.531 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:12.654 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:09.532 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:12.655 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:09.547 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:12.644 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:09.555 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:12.678 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:14.487 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:17.608 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:14.491 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:17.610 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:14.498 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:17.621 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:14.534 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:17.631 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:14.537 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:17.632 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:14.543 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:17.661 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:14.543 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:17.662 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:14.550 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:17.647 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:14.554 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:17.677 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:29.535 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:32.657 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:29.537 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:32.660 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:29.556 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:32.679 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:29.582 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:32.679 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:29.584 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:32.681 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:29.588 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:32.711 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:29.593 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:32.716 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:29.600 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:32.696 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:29.610 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:32.732 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:34.543 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:37.666 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:34.547 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:37.670 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:34.560 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:37.683 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:34.594 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:37.690 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:34.595 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:37.692 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:34.597 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:37.720 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:34.601 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:37.724 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:34.607 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:37.704 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:34.615 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:37.738 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:49.592 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:52.715 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:49.597 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:52.720 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:49.622 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:52.745 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:49.642 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:52.739 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:49.644 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:52.741 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:49.657 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:52.779 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:49.660 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:52.757 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:49.661 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:52.784 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:49.685 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:52.808 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:54.611 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:57.733 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:54.612 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:57.735 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:54.623 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:39:57.746 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:54.658 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:57.754 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:54.658 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:57.755 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:54.662 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:57.785 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:54.664 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:57.787 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:54.668 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:39:57.765 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:31:54.676 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:39:57.799 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:09.656 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:12.778 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:09.660 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:12.783 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:09.677 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:12.800 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:09.704 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:12.800 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:09.705 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:12.802 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:09.711 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:12.833 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:09.781 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:12.816 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:09.781 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:12.835 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:09.781 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:12.852 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:14.668 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:17.791 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:14.813 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:17.796 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:14.813 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:17.811 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:14.813 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:17.814 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:14.813 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:17.817 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:14.813 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:17.828 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:14.814 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:17.846 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:14.814 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:17.849 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:14.814 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:17.860 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:29.845 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:32.967 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:29.847 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:32.970 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:29.862 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:32.985 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:29.894 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:32.990 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:29.895 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:32.992 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:29.898 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:33.021 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:29.899 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:33.022 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:29.910 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:33.006 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:29.914 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:33.037 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:34.749 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:37.872 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:35.033 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:37.874 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:35.034 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:37.884 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:35.034 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:37.893 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:35.034 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:37.894 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:35.034 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:37.904 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:35.038 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:37.922 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:35.038 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:37.925 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:35.039 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:37.939 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:49.798 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:52.921 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:49.801 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:52.923 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:49.819 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:52.942 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:49.846 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:52.943 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:49.848 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:52.945 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:49.850 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:52.973 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:49.852 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:52.975 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:49.868 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:52.965 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:49.882 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:53.003 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:50.326 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:53.423 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:50.619 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:53.474 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:50.619 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:40:53.485 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:50.619 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:53.489 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:50.619 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:53.490 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:50.620 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:40:53.499 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:50.620 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:53.545 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:50.620 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:53.547 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:32:50.620 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:40:53.556 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:09.861 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:41:12.984 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:09.862 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:41:12.985 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:09.881 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:41:13.004 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:09.907 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:41:13.004 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:09.908 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:41:13.005 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:09.926 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:41:13.022 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:09.932 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:41:13.055 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:09.934 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:41:13.057 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:09.953 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:41:13.075 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:10.481 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:41:13.577 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:10.482 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:41:13.578 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:10.492 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:41:13.588 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:10.564 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:41:13.686 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:10.565 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:41:13.687 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:10.573 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:41:13.696 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:10.617 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:41:13.740 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:10.619 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:41:13.741 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:10.626 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:41:13.749 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:29.938 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:41:33.061 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:29.943 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:41:33.066 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:29.989 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:41:33.086 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:29.990 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:41:33.087 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:30.003 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:41:33.126 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:30.013 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:41:33.136 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:30.013 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:41:33.110 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:30.016 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:41:33.139 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:30.042 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:41:33.165 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:30.594 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:41:33.691 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:30.595 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:41:33.692 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:30.603 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:41:33.699 85580== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:30.646 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:41:33.768 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:30.649 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:41:33.772 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:30.657 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:41:33.780 85885== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:30.699 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:41:33.822 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:30.700 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:41:33.823 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:30.712 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:41:33.835 85886== Warning: unimplemented fcntl command: 1036 2023-10-06T14:33:32.127 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T14:33:32.127 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T14:33:32.128 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T14:33:32.151 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T14:33:32.151 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T14:33:32.151 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T14:33:32.152 INFO:tasks.ceph.mgr.x.smithi083.stderr:2023-10-06T14:33:32.150+0000 7f6efbf10640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i x (PID: 85578) UID: 0 2023-10-06T14:33:32.175 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T14:33:32.176 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T14:33:32.176 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T14:33:32.176 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T14:33:32.207 INFO:tasks.ceph.mon.a.smithi019.stderr:2023-10-06T14:33:32.194+0000 9f32640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 85848) UID: 0 2023-10-06T14:33:32.207 INFO:tasks.ceph.mon.a.smithi019.stderr:2023-10-06T14:33:32.196+0000 9f32640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-06T14:33:32.881 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:41:36.004 85885== 2023-10-06T14:33:32.881 INFO:tasks.ceph.mon.a.smithi019.stderr:==00:00:41:36.004 85885== Exit program on first error (--exit-on-first-error=yes) 2023-10-06T14:33:32.901 INFO:tasks.ceph.mon.a.smithi019.stderr:daemon-helper: command failed with exit status 42 2023-10-06T14:33:37.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-10-06T14:33:38.278 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-06T14:33:38.278 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 1463, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 1465, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi019 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2023-10-06T14:33:38.280 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T14:33:38.280 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T14:33:38.280 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T14:33:38.295 INFO:tasks.ceph.mon.c.smithi019.stderr:2023-10-06T14:33:38.283+0000 9f32640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 85850) UID: 0 2023-10-06T14:33:38.297 INFO:tasks.ceph.mon.c.smithi019.stderr:2023-10-06T14:33:38.285+0000 9f32640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-10-06T14:33:38.919 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:41:42.042 85886== 2023-10-06T14:33:38.920 INFO:tasks.ceph.mon.c.smithi019.stderr:==00:00:41:42.042 85886== Exit program on first error (--exit-on-first-error=yes) 2023-10-06T14:33:38.937 INFO:tasks.ceph.mon.c.smithi019.stderr:daemon-helper: command failed with exit status 42 2023-10-06T14:33:43.026 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-10-06T14:33:43.062 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-10-06T14:33:44.382 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-06T14:33:44.392 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 1463, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 1465, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi019 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2023-10-06T14:33:44.393 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T14:33:44.393 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T14:33:44.393 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T14:33:44.407 INFO:tasks.ceph.mon.b.smithi083.stderr:2023-10-06T14:33:44.395+0000 9f32640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 85576) UID: 0 2023-10-06T14:33:44.408 INFO:tasks.ceph.mon.b.smithi083.stderr:2023-10-06T14:33:44.397+0000 9f32640 -1 mon.b@1(probing) e1 *** Got Signal Terminated *** 2023-10-06T14:33:45.000 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:41:48.096 85580== 2023-10-06T14:33:45.000 INFO:tasks.ceph.mon.b.smithi083.stderr:==00:00:41:48.096 85580== Exit program on first error (--exit-on-first-error=yes) 2023-10-06T14:33:45.017 INFO:tasks.ceph.mon.b.smithi083.stderr:daemon-helper: command failed with exit status 42 2023-10-06T14:33:48.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-10-06T14:33:48.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-10-06T14:33:50.495 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-06T14:33:50.495 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 1463, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 1465, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi083 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2023-10-06T14:33:50.496 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T14:33:50.496 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T14:33:50.497 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2023-10-06T14:33:50.539 INFO:teuthology.orchestra.run.smithi019.stdout:2023-10-06T13:57:32.724709+0000 osd.7 (osd.7) 4 : cluster [WRN] osd.7 ep: 149 scrubber::ReplicaReservations pg[14.1]: timeout on replica reservations (since 2023-10-06 13:57:27) 2023-10-06T14:33:50.540 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T14:33:50.540 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2023-10-06T14:33:50.622 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2023-10-06T14:33:50.702 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2023-10-06T14:33:50.781 INFO:teuthology.orchestra.run.smithi019.stdout:2023-10-06T13:57:32.724709+0000 osd.7 (osd.7) 4 : cluster [WRN] osd.7 ep: 149 scrubber::ReplicaReservations pg[14.1]: timeout on replica reservations (since 2023-10-06 13:57:27) 2023-10-06T14:33:50.781 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi019.front.sepia.ceph.com 2023-10-06T14:33:50.781 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T14:33:51.097 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi019.front.sepia.ceph.com 2023-10-06T14:33:51.098 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T14:33:51.166 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi019.front.sepia.ceph.com 2023-10-06T14:33:51.166 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T14:33:51.233 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi019.front.sepia.ceph.com 2023-10-06T14:33:51.233 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T14:33:51.281 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi083.front.sepia.ceph.com 2023-10-06T14:33:51.281 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T14:33:51.367 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi083.front.sepia.ceph.com 2023-10-06T14:33:52.215 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T14:33:52.291 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi083.front.sepia.ceph.com 2023-10-06T14:33:52.829 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T14:33:52.900 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi083.front.sepia.ceph.com 2023-10-06T14:33:52.984 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T14:33:53.049 INFO:tasks.ceph:Archiving mon data... 2023-10-06T14:33:53.050 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609/data/mon.a.tgz 2023-10-06T14:33:53.050 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2023-10-06T14:33:53.069 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.AsWYztfw7p 2023-10-06T14:33:53.069 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.AsWYztfw7p 2023-10-06T14:33:53.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.AsWYztfw7p 2023-10-06T14:33:53.303 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-10-06T14:33:53.394 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.AsWYztfw7p is 507KB 2023-10-06T14:33:53.559 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.AsWYztfw7p 2023-10-06T14:33:53.615 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609/data/mon.c.tgz 2023-10-06T14:33:53.635 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2023-10-06T14:33:53.670 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.N0dBUqHjII 2023-10-06T14:33:53.671 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.N0dBUqHjII 2023-10-06T14:33:53.831 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.N0dBUqHjII 2023-10-06T14:33:53.919 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.N0dBUqHjII is 505KB 2023-10-06T14:33:54.045 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.N0dBUqHjII 2023-10-06T14:33:54.101 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609/data/mon.b.tgz 2023-10-06T14:33:54.101 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2023-10-06T14:33:54.115 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.TFrbQZMad2 2023-10-06T14:33:54.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.TFrbQZMad2 2023-10-06T14:33:54.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.TFrbQZMad2 2023-10-06T14:33:54.381 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.TFrbQZMad2 is 505KB 2023-10-06T14:33:54.490 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.TFrbQZMad2 2023-10-06T14:33:54.538 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T14:33:54.539 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T14:33:54.541 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T14:33:54.603 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T14:33:54.604 DEBUG:teuthology.orchestra.run.smithi019:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T14:33:54.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T14:33:54.661 INFO:teuthology.orchestra.run.smithi019.stdout:/var/log/ceph/valgrind/mon.a.log: Leak_StillReachable 2023-10-06T14:33:54.662 INFO:teuthology.orchestra.run.smithi019.stdout:/var/log/ceph/valgrind/mon.c.log: Leak_StillReachable 2023-10-06T14:33:54.662 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/mon.a.log kind Leak_StillReachable 2023-10-06T14:33:54.662 ERROR:tasks.ceph:saw valgrind issue Leak_StillReachable in /var/log/ceph/valgrind/mon.a.log 2023-10-06T14:33:54.662 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/mon.c.log kind Leak_StillReachable 2023-10-06T14:33:54.662 ERROR:tasks.ceph:saw valgrind issue Leak_StillReachable in /var/log/ceph/valgrind/mon.c.log 2023-10-06T14:33:54.682 INFO:teuthology.orchestra.run.smithi083.stdout:/var/log/ceph/valgrind/mon.b.log: Leak_StillReachable 2023-10-06T14:33:54.682 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/mon.b.log kind Leak_StillReachable 2023-10-06T14:33:54.682 ERROR:tasks.ceph:saw valgrind issue Leak_StillReachable in /var/log/ceph/valgrind/mon.b.log 2023-10-06T14:33:54.683 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T14:33:54.684 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609/remote/smithi019/crash 2023-10-06T14:33:54.685 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T14:33:54.717 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609/remote/smithi083/crash 2023-10-06T14:33:54.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T14:33:54.748 INFO:tasks.ceph:Compressing logs... 2023-10-06T14:33:54.748 DEBUG:teuthology.orchestra.run.smithi019:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T14:33:54.762 DEBUG:teuthology.orchestra.run.smithi083:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T14:34:19.154 INFO:tasks.ceph:Archiving logs... 2023-10-06T14:34:19.154 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609/remote/smithi019/log 2023-10-06T14:34:19.155 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T14:34:22.111 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609/remote/smithi083/log 2023-10-06T14:34:22.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T14:34:24.455 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 322, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 1465, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 1951, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 362, in valgrind_post raise valgrind_exception Exception: saw valgrind issues 2023-10-06T14:34:24.519 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T14:34:24.528 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 322, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 1465, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 1951, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_4af54c4a1461e3792e9783e337bb62ce429214f3/qa/tasks/ceph.py", line 362, in valgrind_post raise valgrind_exception Exception: saw valgrind issues 2023-10-06T14:34:24.529 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T14:34:24.529 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T14:34:24.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T14:34:24.614 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T14:34:24.623 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T14:34:24.623 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T14:34:24.626 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T14:34:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2023-10-06T14:34:24.639 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2023-10-06T14:34:24.642 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T14:34:24.642 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2023-10-06T14:34:24.643 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv01.front.sepia.ceph.com 2 9 377 479 +148us[ +148us] +/- 80ms 2023-10-06T14:34:24.643 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv02.front.sepia.ceph.com 3 9 377 482 -580us[ -579us] +/- 58ms 2023-10-06T14:34:24.643 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 95 +543us[ +543us] +/- 83ms 2023-10-06T14:34:24.643 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T14:34:24.644 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T14:34:24.644 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2023-10-06T14:34:24.644 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 +151us[ +115us] +/- 71ms 2023-10-06T14:34:24.644 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -545us[ -580us] +/- 49ms 2023-10-06T14:34:24.644 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 92 +645us[ +626us] +/- 83ms 2023-10-06T14:34:24.644 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T14:34:24.645 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T14:34:24.655 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T14:34:24.655 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T14:34:24.682 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T14:34:24.792 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T14:34:24.883 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T14:34:24.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T14:34:24.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T14:34:24.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T14:34:24.951 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T14:34:24.962 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T14:34:24.999 INFO:teuthology.task.internal:Duration was 3463.727906 seconds 2023-10-06T14:34:25.000 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T14:34:25.033 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T14:34:25.033 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T14:34:25.036 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T14:34:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T14:34:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T14:34:25.477 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T14:34:25.477 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2023-10-06T14:34:25.478 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T14:34:25.498 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2023-10-06T14:34:25.537 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T14:34:25.559 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T14:34:25.589 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T14:34:25.592 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T14:34:25.626 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T14:34:25.629 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T14:34:25.665 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T14:34:25.776 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T14:34:25.802 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T14:34:25.803 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T14:34:25.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T14:34:25.904 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T14:34:25.914 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T14:34:25.916 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T14:34:25.943 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2023-10-06T14:34:26.057 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2023-10-06T14:34:26.100 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T14:34:26.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T14:34:26.129 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T14:34:26.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T14:34:26.154 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T14:34:26.164 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T14:34:26.189 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609/remote/smithi019 2023-10-06T14:34:26.189 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T14:34:26.229 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609/remote/smithi083 2023-10-06T14:34:26.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T14:34:26.285 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T14:34:26.285 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T14:34:26.287 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T14:34:26.332 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T14:34:26.358 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T14:34:26.370 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T14:34:26.381 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T14:34:26.402 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T14:34:26.405 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T14:34:26.417 INFO:teuthology.orchestra.run.smithi019.stdout: 265242 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 14:34 /home/ubuntu/cephtest 2023-10-06T14:34:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: 265982 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 14:34 /home/ubuntu/cephtest 2023-10-06T14:34:26.424 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T14:34:26.434 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T14:34:26.458 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: Exception: saw valgrind issues 2023-10-06T14:34:26.458 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T14:34:26.484 DEBUG:teuthology.nuke:shortname: smithi019 2023-10-06T14:34:26.503 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T14:34:26.529 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 13:28:40.481228', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKFVHDP42vImDPuf04uS5rW92D9ajR1cl+iGPKEkXHSvOkbk7aW87koA584qj1YGCwIYmkWLzgrQm43lX51y5DE='} 2023-10-06T14:34:26.533 DEBUG:teuthology.nuke:shortname: smithi083 2023-10-06T14:34:26.534 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T14:34:26.553 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:58:21-rados-wip-yuri7-testing-2023-10-04-1350-reef-distro-default-smithi/7412609', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 13:28:40.479983', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPAnIPUznwUocjlTpOrgy72Ol5ISFYjF2B2TswcYhFAoqtpeWOZpkDxB8ufWw295eDlKNnYynEE4qzouoO9i7dk='} 2023-10-06T14:34:26.594 INFO:teuthology.orchestra.console.smithi019:Power off 2023-10-06T14:34:26.594 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T14:34:26.620 DEBUG:teuthology.orchestra.console.smithi019:power off output: Chassis Power Control: Down/Off 2023-10-06T14:34:26.621 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T14:34:26.639 INFO:teuthology.orchestra.console.smithi083:Power off 2023-10-06T14:34:26.639 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T14:34:26.649 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2023-10-06T14:34:26.665 DEBUG:teuthology.orchestra.console.smithi083:power off output: Chassis Power Control: Down/Off 2023-10-06T14:34:26.665 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T14:34:26.692 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2023-10-06T14:34:30.650 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T14:34:30.694 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T14:34:30.786 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2023-10-06T14:34:30.804 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2023-10-06T14:34:34.030 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-10-06T14:34:34.788 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T14:34:34.806 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T14:34:34.898 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2023-10-06T14:34:34.916 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2023-10-06T14:34:38.902 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T14:34:38.916 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T14:34:39.012 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2023-10-06T14:34:39.025 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2023-10-06T14:34:39.032 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~5s 2023-10-06T14:34:39.114 INFO:teuthology.orchestra.console.smithi019:Power off completed 2023-10-06T14:34:39.126 INFO:teuthology.orchestra.console.smithi083:Power off completed 2023-10-06T14:34:39.230 INFO:teuthology.run:Summary data: description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} duration: 3463.7279064655304 failure_reason: '"2023-10-06T13:57:32.724709+0000 osd.7 (osd.7) 4 : cluster [WRN] osd.7 ep: 149 scrubber::ReplicaReservations pg[14.1]: timeout on replica reservations (since 2023-10-06 13:57:27)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2023-10-06T14:34:39.231 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T14:34:39.379 INFO:teuthology.run:FAIL